-- Logs begin at Sat 2022-08-06 13:48:45 UTC, end at Sat 2022-08-06 14:56:56 UTC. -- Aug 06 14:16:10.855146 dv-cinder-851996-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 06 14:16:10.863614 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 6 14:16:10 2022] *** Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: nodename: dv-cinder-851996-4-iscsi Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: machine: x86_64 Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: clock source: unix Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: pcre jit disabled Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: detected number of CPU cores: 4 Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: current working directory: / Aug 06 14:16:10.865608 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: detected binary path: /usr/bin/uwsgi-core Aug 06 14:16:10.866111 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: your processes number limit is 47680 Aug 06 14:16:10.866111 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: your memory page size is 4096 bytes Aug 06 14:16:10.866111 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: detected max file descriptor number: 2048 Aug 06 14:16:10.866111 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: lock engine: pthread robust mutexes Aug 06 14:16:10.866111 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: thunder lock: enabled Aug 06 14:16:10.866111 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 06 14:16:10.866111 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 06 14:16:10.905175 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: Python main interpreter initialized at 0x559f1f0959c0 Aug 06 14:16:10.905175 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: python threads support enabled Aug 06 14:16:10.905175 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: your server socket listen backlog is limited to 100 connections Aug 06 14:16:10.905175 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: your mercy for graceful operations on workers is 90 seconds Aug 06 14:16:10.905516 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 14:16:10.905590 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: *** Operational MODE: preforking *** Aug 06 14:16:10.905850 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: *** uWSGI is running in multiple interpreter mode *** Aug 06 14:16:10.905850 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: spawned uWSGI master process (pid: 80138) Aug 06 14:16:10.906043 dv-cinder-851996-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 06 14:16:10.910894 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: spawned uWSGI worker 1 (pid: 80150, cores: 1) Aug 06 14:16:10.910894 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: spawned uWSGI worker 2 (pid: 80151, cores: 1) Aug 06 14:16:10.910894 dv-cinder-851996-4-iscsi devstack@keystone.service[80138]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 14:16:16.414487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 14:16:16.414487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: from cryptography.utils import int_from_bytes Aug 06 14:16:16.415284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 14:16:16.415284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: from cryptography.utils import int_from_bytes Aug 06 14:16:16.418386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 14:16:16.418386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: from cryptography.utils import int_from_bytes Aug 06 14:16:16.418840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 14:16:16.418840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: from cryptography.utils import int_from_bytes Aug 06 14:16:16.465090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 14:16:16.465283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 14:16:16.465475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] command line args: [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 14:16:16.465751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 14:16:16.465828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 14:16:16.465929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ================================================================================ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 14:16:16.466013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 14:16:16.466128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] command line args: [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 14:16:16.466205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] admin_token = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.466395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 14:16:16.466472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config_dir = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.466573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ================================================================================ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 14:16:16.466648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.466753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config_source = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.466829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] admin_token = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.466957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] debug = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.467081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config_dir = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.467157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.467466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.467466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config_source = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.467692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.467811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] debug = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.467925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.468890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_config_append = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.469007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.469125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.469244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.469363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_dir = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.469478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_config_append = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.469844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.469963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_dir = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.470896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.471935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.472976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] publish_errors = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] publish_errors = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.473992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] standard_threads = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] standard_threads = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.474882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_journal = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_json = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_stderr = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_journal = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_syslog = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_json = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_stderr = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.475983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_syslog = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.476060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.476212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 14:16:16.476295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.476465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.476549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.476701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.476785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.476961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477044 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.477995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.478997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.479938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.480904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.481882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.482979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.483983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.484921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.485978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.486913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.487919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.488997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.489985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.490938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491632 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.491861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.492976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.493055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.493211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.493291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.493445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.493531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.493676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.493752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.494422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.494507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.494657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.494745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.494905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.494997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.495911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.496864 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.497938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.498984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.499101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.499191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.499332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.499410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.499830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.499916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.499987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500288 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.500955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.external = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.external = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.password = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.501998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.502149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.token = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.502228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.502376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.password = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.502459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.502604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.token = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.502682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.502839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.503877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.504971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.505990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.506166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.506244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.506417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.506495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.506650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.506728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.506933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.507933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.508950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.509120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.509207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.509347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.509426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.509662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.510237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.510379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.510457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.510632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.510712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.510877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.510958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.511996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.512842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.513940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.514976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.515951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.516901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.517974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.518954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.519962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.520888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.521974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.522941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523865 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.523945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.524103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.524181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.524348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.524424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.525956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.526993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.527976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.528920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.529867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.530976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531450 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.531919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.532998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.533957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.534990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.535867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.536840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.537961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.538047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.538206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.538282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.538449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.538609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.538686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.538847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.539912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.540887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.541966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.542948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543722 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.543959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.544916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545622 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.545860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.546975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.547981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.548058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.548217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.548304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.548457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.548544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.548691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.548769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.549969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.550931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.551873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.552022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.552100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.552191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 14:16:16.552278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.552475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 14:16:16.552643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 14:16:16.581760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.581977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.582083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.582201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.582283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.582417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.590997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.591173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.620766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.620906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.620971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.621042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.672748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.672919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.722986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.723217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.723528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.723764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.787221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.787367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.787453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.787598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.787682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.787818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.787903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.788032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.788115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.788245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.788327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.788455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.812724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.812844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.812955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.813907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.814032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.814115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.825534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.825635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.838690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.838860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.838964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.839882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.840011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.840091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.840216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 14:16:16.857038 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.857209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.857316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 14:16:16.857424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 14:16:16.857578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.857763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.857848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 14:16:16.857954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 14:16:16.858036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.858221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.858331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.858476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.858557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.858727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.858807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.859001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.868385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.868516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.868663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.868789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.868961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.869079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.869244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.869356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.876114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.876210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.880837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.880943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.881116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.881199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.881389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.881478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.881663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.881744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.881936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.882016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.882192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.882281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.882450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.882531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.882717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.882796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.883003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.883096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.883271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.883352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.883530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.883611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.907202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.907481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.908664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.908941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.909021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.909222 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.913495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.913770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.913853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.914046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.914129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.914350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.920702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.920965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.921062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.921246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.925323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.925560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.925642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.925837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.925918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.926114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.926196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.926394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.926476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.926671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.926761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.926969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.927054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.927235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.927315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.927523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.942937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.943213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.943352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.943627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.947605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.948087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.950700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.951013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.951221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.951515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.955349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.955902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.958555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.959283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.961715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.962009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.962290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.962414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.962571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.963286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.963413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.963581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.963686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.963863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.964140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.964426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.979470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.979747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.980009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.980278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.980359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.980614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.980879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.981151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.987059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.988001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.988602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.988882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.989535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.989809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.991685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.991963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.992289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.992585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.992666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:16.992859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.992941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.993194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.993473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:16.993747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.004167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.004445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.004720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.004996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.005128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.005270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.005407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.005539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.005685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.005819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.005963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.006242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.006516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.006803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.021770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.022904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.024721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.026423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.028243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.029403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.030712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.031904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.032235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.032513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.032805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.033449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.033728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.034018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.039188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.040456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.040800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.042079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.043741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.044032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.044312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.044583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.045015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.045305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.045588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.045862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.052784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.053074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.053351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.054147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.054436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.054725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.060673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.060950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.061226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.061496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.061759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.062031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.062112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 14:16:17.062315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.062406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.062585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.062667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.062917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.063001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.063193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.063287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.063463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.063543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.063730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.063862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.064005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.064087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.064311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.064393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.064639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.064911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.065171 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.065444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.065711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 14:16:17.097588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559f1f0959c0 pid: 80150 (default app) Aug 06 14:16:17.099368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559f1f0959c0 pid: 80151 (default app) Aug 06 14:16:17.100831 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ce3a6-f4ff-4ccc-808d-87dac1425461 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:17.101119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ce3a6-f4ff-4ccc-808d-87dac1425461 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:17.101390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ce3a6-f4ff-4ccc-808d-87dac1425461 None None] PATH_INFO: `/v3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:17.105672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Sat Aug 6 14:16:17 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:24.621721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca417d2-ee26-459a-9e88-d47fbb3cdd48 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:24.622239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca417d2-ee26-459a-9e88-d47fbb3cdd48 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:24.622239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca417d2-ee26-459a-9e88-d47fbb3cdd48 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:24.626622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:24.632038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:24.632513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:24.632622 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:24.702694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: INFO dbcounter [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] Registered counter for database keystone Aug 06 14:16:24.721305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG oslo_db.sqlalchemy.engines [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80150) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 14:16:24.727417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writer thread running {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 14:16:24.904300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG passlib.handlers.bcrypt [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80150) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 14:16:24.904730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG passlib.handlers.bcrypt [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80150) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 14:16:24.958733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:24.960329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:25.087635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 06 14:16:25.219352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca417d2-ee26-459a-9e88-d47fbb3cdd48 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:25.288397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: INFO dbcounter [None req-9ca417d2-ee26-459a-9e88-d47fbb3cdd48 None None] Registered counter for database keystone Aug 06 14:16:25.310050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG oslo_db.sqlalchemy.engines [None req-9ca417d2-ee26-459a-9e88-d47fbb3cdd48 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80151) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 14:16:25.316098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writer thread running {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 14:16:25.432508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca417d2-ee26-459a-9e88-d47fbb3cdd48 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:25.436794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e0538-e34c-4d41-9071-5653b75e5ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:25.437425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e0538-e34c-4d41-9071-5653b75e5ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:25.437708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e0538-e34c-4d41-9071-5653b75e5ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:25.437985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e0538-e34c-4d41-9071-5653b75e5ed1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:25.445909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e0538-e34c-4d41-9071-5653b75e5ed1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:25.498532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a11e0538-e34c-4d41-9071-5653b75e5ed1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 14:16:25.498817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e0538-e34c-4d41-9071-5653b75e5ed1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:25.508168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Sat Aug 6 14:16:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:16:25.514137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:25.529048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765f96c-d364-4632-85c2-a266b67dc11a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:25.533506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7fd7e-97dd-427c-b33f-83b192c459b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:25.534621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fd7e-97dd-427c-b33f-83b192c459b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:25.534934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fd7e-97dd-427c-b33f-83b192c459b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:25.535222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fd7e-97dd-427c-b33f-83b192c459b0 admin admin] PATH_INFO: `/v3/projects/3509ae96075e47edbbf7f243bb4b1186` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:25.536733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7fd7e-97dd-427c-b33f-83b192c459b0 admin admin] RBAC: Authorizing `identity:get_project(project_id=3509ae96075e47edbbf7f243bb4b1186)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:25.588023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING keystone.common.rbac_enforcer.enforcer [None req-13e7fd7e-97dd-427c-b33f-83b192c459b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 14:16:25.588365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7fd7e-97dd-427c-b33f-83b192c459b0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:25.592973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Sat Aug 6 14:16:25 2022] GET /identity/v3/projects/3509ae96075e47edbbf7f243bb4b1186 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:27.564533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c99a4-ebbc-4cf4-9c92-1736b258f520 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:27.565078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c99a4-ebbc-4cf4-9c92-1736b258f520 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:27.565078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c99a4-ebbc-4cf4-9c92-1736b258f520 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:27.569245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:27.574632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8dc3-c8ee-426e-afdc-c8bb74cdd5e2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:27.575196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8dc3-c8ee-426e-afdc-c8bb74cdd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:27.575281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8dc3-c8ee-426e-afdc-c8bb74cdd5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:27.595091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-843f8dc3-c8ee-426e-afdc-c8bb74cdd5e2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:27.596611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-843f8dc3-c8ee-426e-afdc-c8bb74cdd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:27.605055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 14:16:27.626308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c99a4-ebbc-4cf4-9c92-1736b258f520 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:27.638297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c99a4-ebbc-4cf4-9c92-1736b258f520 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:27.642440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a5aed-c506-4a25-9251-c0ad836e3083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:27.643085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5aed-c506-4a25-9251-c0ad836e3083 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:27.643369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5aed-c506-4a25-9251-c0ad836e3083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:27.643643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5aed-c506-4a25-9251-c0ad836e3083 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:27.651807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a5aed-c506-4a25-9251-c0ad836e3083 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:27.653557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a5aed-c506-4a25-9251-c0ad836e3083 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:27.662120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Sat Aug 6 14:16:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:16:27.667748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f8dc3-c8ee-426e-afdc-c8bb74cdd5e2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:27.676814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f8dc3-c8ee-426e-afdc-c8bb74cdd5e2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:27.681182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f50368-8ae7-40e1-9292-dfd1c244be52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:27.682080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f50368-8ae7-40e1-9292-dfd1c244be52 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:27.682358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f50368-8ae7-40e1-9292-dfd1c244be52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:27.682629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f50368-8ae7-40e1-9292-dfd1c244be52 admin admin] PATH_INFO: `/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:27.684005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f50368-8ae7-40e1-9292-dfd1c244be52 admin admin] RBAC: Authorizing `identity:get_user(user_id=b7c86189492e4a4d8e62d6e2879b6ae2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:27.685795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f50368-8ae7-40e1-9292-dfd1c244be52 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:27.687357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Sat Aug 6 14:16:27 2022] GET /identity/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:29.735968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0b16d-67ac-4bb1-bcf5-41a9e5e45052 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:29.736506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0b16d-67ac-4bb1-bcf5-41a9e5e45052 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:29.736506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0b16d-67ac-4bb1-bcf5-41a9e5e45052 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:29.737849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:29.742809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2d38-4d14-455b-90ad-b32b691ff6d1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:29.743222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2d38-4d14-455b-90ad-b32b691ff6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:29.743391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2d38-4d14-455b-90ad-b32b691ff6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:29.752742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f55cc-32ab-4919-be74-f18fb7636337 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:29.753023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f55cc-32ab-4919-be74-f18fb7636337 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:29.753300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f55cc-32ab-4919-be74-f18fb7636337 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:29.754601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:29.759579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:29.759865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:29.760142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:29.763449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bfdd2d38-4d14-455b-90ad-b32b691ff6d1 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:29.764628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bfdd2d38-4d14-455b-90ad-b32b691ff6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:29.801883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG passlib.handlers.bcrypt [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80151) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 14:16:29.802325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG passlib.handlers.bcrypt [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80151) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 14:16:29.847663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:29.847878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:29.849271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:29.870313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd2d38-4d14-455b-90ad-b32b691ff6d1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:29.895434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:29.948773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:29.970111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:29.976225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd2d38-4d14-455b-90ad-b32b691ff6d1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.009086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.009666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.009948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.010224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.021167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:30.030444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:30.031435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:30.032273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:16:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:16:30.038240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:30.070153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25ad8c-1520-4155-b403-91130de90dac None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.073035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:30.110057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.110941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.111230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.111505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.119548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:30.138252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f1205-f1cf-49bf-b094-f1d27ec657c1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.171758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.172371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.172650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.172923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.173635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:30.173941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:30.175611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:30.178000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:30.179616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:30.188466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:30.194099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:30.279798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 14:16:30.285689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:30.289538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:30.385521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a14d4-3cb0-46fe-893b-1aa29532ad3b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.397561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab408f67-d368-441f-8a16-cdbf76badc65 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.421614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.422477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.422755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.423067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.435770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.436430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.436716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.436995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] PATH_INFO: `/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.438176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c86189492e4a4d8e62d6e2879b6ae2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:30.439688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:30.441498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:30.441607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:30 2022] GET /identity/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:30.442647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:30.447225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:16:30.461076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:30.461937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:30.463334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:16:30.463334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:16:30.463334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:16:30.463334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:30.463531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:30.463595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:16:30.468578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:30.522446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:30.549549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec7138-3c91-4c1e-8d39-a78b68f2bc35 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.567190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959539dd-10d9-45a1-a37c-461c3de52767 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.583004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.583659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.583946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.584227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.585396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:30.586946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:30.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:30.594094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:30.600629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43804e38-49d9-43f6-ae33-981f5be807ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.601246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43804e38-49d9-43f6-ae33-981f5be807ac None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.601522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43804e38-49d9-43f6-ae33-981f5be807ac None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.601790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43804e38-49d9-43f6-ae33-981f5be807ac None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.602477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43804e38-49d9-43f6-ae33-981f5be807ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:30.602778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43804e38-49d9-43f6-ae33-981f5be807ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:30.604213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43804e38-49d9-43f6-ae33-981f5be807ac None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:30.621798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:30.678725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:30.700796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1e9ab-a50f-4d29-8bb3-859eac2bcb8d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:30.740797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b1701-c845-4215-89be-93c26289ad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:30.741435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b1701-c845-4215-89be-93c26289ad8e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:30.741721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b1701-c845-4215-89be-93c26289ad8e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:30.742003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b1701-c845-4215-89be-93c26289ad8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:30.742811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b1701-c845-4215-89be-93c26289ad8e None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.domain.id=default, user.id=b7c86189492e4a4d8e62d6e2879b6ae2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:30.743158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b1701-c845-4215-89be-93c26289ad8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:30.745127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b1701-c845-4215-89be-93c26289ad8e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:30.752698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Sat Aug 6 14:16:30 2022] GET /identity/v3/role_assignments?user.id=b7c86189492e4a4d8e62d6e2879b6ae2&scope.domain.id=default&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:33.154169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032562a0-97a0-44ac-bd0f-2b5c91007832 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.155221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032562a0-97a0-44ac-bd0f-2b5c91007832 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.155715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032562a0-97a0-44ac-bd0f-2b5c91007832 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.157305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:33.162711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f555a-e155-4758-98d3-befccd8f6852 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.163544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f555a-e155-4758-98d3-befccd8f6852 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.164057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f555a-e155-4758-98d3-befccd8f6852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.185295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ca3f555a-e155-4758-98d3-befccd8f6852 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:33.186841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ca3f555a-e155-4758-98d3-befccd8f6852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:33.220847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590d0e-19aa-4fcb-a5cd-d29cb9a014e4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.221439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590d0e-19aa-4fcb-a5cd-d29cb9a014e4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.221936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590d0e-19aa-4fcb-a5cd-d29cb9a014e4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.223491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:33.229273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fefd1-4ac0-4b9f-953b-cbd611d146ad None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.229797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fefd1-4ac0-4b9f-953b-cbd611d146ad None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.230283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fefd1-4ac0-4b9f-953b-cbd611d146ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.266073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d95fefd1-4ac0-4b9f-953b-cbd611d146ad None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:33.268859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d95fefd1-4ac0-4b9f-953b-cbd611d146ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:33.337751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:33.374634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f555a-e155-4758-98d3-befccd8f6852 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:33.418163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:33.461595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fefd1-4ac0-4b9f-953b-cbd611d146ad None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:33.492889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:33.530139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f555a-e155-4758-98d3-befccd8f6852 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:33.552684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:33.606562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ed391-2ad8-4594-b746-2d57f8412a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:33.607634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed391-2ad8-4594-b746-2d57f8412a85 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.608123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed391-2ad8-4594-b746-2d57f8412a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.608599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed391-2ad8-4594-b746-2d57f8412a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.625721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ed391-2ad8-4594-b746-2d57f8412a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:33.627673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ed391-2ad8-4594-b746-2d57f8412a85 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:33.648134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fefd1-4ac0-4b9f-953b-cbd611d146ad None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-533ed391-2ad8-4594-b746-2d57f8412a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:16:33.652697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:33.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:33.654514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:16:33.654514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:16:33.654514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:16:33.654514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:33.656057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 13/26] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:16:33.657403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7cae8-4842-4373-b250-01acbdfe1c5b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.657848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7cae8-4842-4373-b250-01acbdfe1c5b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.658290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7cae8-4842-4373-b250-01acbdfe1c5b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.660272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 14/27] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:33.661541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e82c0-f472-4faf-9750-c450ad27d0c4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.661957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e82c0-f472-4faf-9750-c450ad27d0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.662358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e82c0-f472-4faf-9750-c450ad27d0c4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.663900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 15/28] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:33.668963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e82c0-f472-4faf-9750-c450ad27d0c4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:33.697185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe1da5-bf1e-4a18-9f52-e8bdd76b42e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:33.698158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1da5-bf1e-4a18-9f52-e8bdd76b42e3 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.698646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1da5-bf1e-4a18-9f52-e8bdd76b42e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.699170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1da5-bf1e-4a18-9f52-e8bdd76b42e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.700100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe1da5-bf1e-4a18-9f52-e8bdd76b42e3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:33.701701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe1da5-bf1e-4a18-9f52-e8bdd76b42e3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:33.781755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e82c0-f472-4faf-9750-c450ad27d0c4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:33.812606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:33.823066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0218d9-c0ba-4ed4-8a4b-bf42ac083654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:33.823947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0218d9-c0ba-4ed4-8a4b-bf42ac083654 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.824401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0218d9-c0ba-4ed4-8a4b-bf42ac083654 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.824827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0218d9-c0ba-4ed4-8a4b-bf42ac083654 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.825682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0218d9-c0ba-4ed4-8a4b-bf42ac083654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:33.826128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0218d9-c0ba-4ed4-8a4b-bf42ac083654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:33.827767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0218d9-c0ba-4ed4-8a4b-bf42ac083654 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:33.858697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 16/29] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:33.860498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeba13b-2c6f-43e1-b8eb-d6eddeb719db None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.860946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeba13b-2c6f-43e1-b8eb-d6eddeb719db None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.861361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeba13b-2c6f-43e1-b8eb-d6eddeb719db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.880570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0aeba13b-2c6f-43e1-b8eb-d6eddeb719db None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:33.881956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0aeba13b-2c6f-43e1-b8eb-d6eddeb719db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:33.945504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 14/30] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Aug 6 14:16:33 2022] POST /identity/v3/roles => generated 219 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:16:33.947448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85549c-4e25-4f3b-8426-7888807a2cd6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:33.947980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85549c-4e25-4f3b-8426-7888807a2cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:33.948493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85549c-4e25-4f3b-8426-7888807a2cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:33.954126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 17/31] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:33.955311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeba13b-2c6f-43e1-b8eb-d6eddeb719db None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:33.968948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-be85549c-4e25-4f3b-8426-7888807a2cd6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:33.970728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-be85549c-4e25-4f3b-8426-7888807a2cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:34.047439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 15/32] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:34.048461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85549c-4e25-4f3b-8426-7888807a2cd6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.069253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeba13b-2c6f-43e1-b8eb-d6eddeb719db None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:34.091153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:34.094136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:34.114129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7372fcb-585f-4b9d-a9e6-635e10ccfe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:34.115222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7372fcb-585f-4b9d-a9e6-635e10ccfe34 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.116321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7372fcb-585f-4b9d-a9e6-635e10ccfe34 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.116851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7372fcb-585f-4b9d-a9e6-635e10ccfe34 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.117781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7372fcb-585f-4b9d-a9e6-635e10ccfe34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:34.120861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7372fcb-585f-4b9d-a9e6-635e10ccfe34 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:34.174913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85549c-4e25-4f3b-8426-7888807a2cd6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:34.217964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:34.218887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.219387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.219807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.220771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:34.222587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:34.239336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 16/33] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Aug 6 14:16:34 2022] POST /identity/v3/roles => generated 225 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:34.240378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.279758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 18/34] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:16:33 2022] POST /identity/v3/projects => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:34.281839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4dfaa-d8ae-41d2-85e9-2ea7706700f6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.282244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4dfaa-d8ae-41d2-85e9-2ea7706700f6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.282660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4dfaa-d8ae-41d2-85e9-2ea7706700f6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.284266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 19/35] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:34.293899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d44bc-cfd0-4785-a421-71a699712c4d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.293899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d44bc-cfd0-4785-a421-71a699712c4d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.293899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d44bc-cfd0-4785-a421-71a699712c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.315003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f25d44bc-cfd0-4785-a421-71a699712c4d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:34.316656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f25d44bc-cfd0-4785-a421-71a699712c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:34.427024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 20/36] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:34.445742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:34.464917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37879d9-f591-4f8e-9855-4252229fa88e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:34.489049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d44bc-cfd0-4785-a421-71a699712c4d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.515781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:34.516547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.516836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.517113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.532831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:34.539828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:34.540732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:34.541482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 17/37] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:16:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:34.548631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.613531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:34.619353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d44bc-cfd0-4785-a421-71a699712c4d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:34.656880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:34.657551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.657839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.658116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.659289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:34.660872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:34.662910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 21/38] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:34.666366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892993b5-bda2-45c1-b344-9edfc3d02739 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:34.669597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.705042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcd20f-3cd8-40b6-9841-47db68419cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:34.705668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcd20f-3cd8-40b6-9841-47db68419cab None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.705951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcd20f-3cd8-40b6-9841-47db68419cab None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.706231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcd20f-3cd8-40b6-9841-47db68419cab None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.706988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcd20f-3cd8-40b6-9841-47db68419cab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:34.707298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcd20f-3cd8-40b6-9841-47db68419cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:34.708692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcd20f-3cd8-40b6-9841-47db68419cab None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:34.721741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 18/39] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:34.722779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9e35b-4539-4c80-8ab9-ee2db723f19d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.723103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9e35b-4539-4c80-8ab9-ee2db723f19d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.723392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9e35b-4539-4c80-8ab9-ee2db723f19d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.726906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 19/40] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:34.726906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9e35b-4539-4c80-8ab9-ee2db723f19d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.746837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:34.774885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06052-d89a-4445-81ce-5b883bb8cd1d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:34.808232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53243db7-6db3-4225-83df-4a09fe99242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:34.808854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53243db7-6db3-4225-83df-4a09fe99242d None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.809138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53243db7-6db3-4225-83df-4a09fe99242d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.809413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53243db7-6db3-4225-83df-4a09fe99242d None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.810100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53243db7-6db3-4225-83df-4a09fe99242d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:34.811593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53243db7-6db3-4225-83df-4a09fe99242d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:34.828987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9e35b-4539-4c80-8ab9-ee2db723f19d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:34.838265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 22/41] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:16:34 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:34.840249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabace12-d3d5-4be7-a842-8a51d326491b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.840598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabace12-d3d5-4be7-a842-8a51d326491b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.840929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabace12-d3d5-4be7-a842-8a51d326491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.861136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-fabace12-d3d5-4be7-a842-8a51d326491b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:34.862512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fabace12-d3d5-4be7-a842-8a51d326491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:34.873011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:34.873689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:34.873975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:34.874255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] PATH_INFO: `/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:34.876115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c86189492e4a4d8e62d6e2879b6ae2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:34.877597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:34.879605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 20/42] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:34 2022] GET /identity/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:34.880370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:34.885101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.947916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 23/43] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:34.970524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabace12-d3d5-4be7-a842-8a51d326491b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:34.988737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:35.007654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731a21f-c458-412b-bbc4-2a426c6b5004 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:35.051731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:35.052381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:35.052660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:35.052934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:35.054086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:35.055871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:35.057482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 21/44] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:35.063024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:35.081352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabace12-d3d5-4be7-a842-8a51d326491b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:35.118054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f2d45-b9e2-479d-b58f-2ceb864f6ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:35.118721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2d45-b9e2-479d-b58f-2ceb864f6ff0 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:35.119036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2d45-b9e2-479d-b58f-2ceb864f6ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:35.119319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2d45-b9e2-479d-b58f-2ceb864f6ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:35.120044 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f2d45-b9e2-479d-b58f-2ceb864f6ff0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:35.121442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f2d45-b9e2-479d-b58f-2ceb864f6ff0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:35.137975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 24/45] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Aug 6 14:16:34 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:16:35.172454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285497f8-7708-4c4a-b402-d523f3ccd035 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:35.210493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3a7a5-b808-48a6-9c4a-896e18c6b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:35.211301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3a7a5-b808-48a6-9c4a-896e18c6b590 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:35.211493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3a7a5-b808-48a6-9c4a-896e18c6b590 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:35.211772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3a7a5-b808-48a6-9c4a-896e18c6b590 None admin] PATH_INFO: `/v3/domains/default/users/b7c86189492e4a4d8e62d6e2879b6ae2/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:35.214377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3a7a5-b808-48a6-9c4a-896e18c6b590 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b7c86189492e4a4d8e62d6e2879b6ae2, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:35.215909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3a7a5-b808-48a6-9c4a-896e18c6b590 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:35.232520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 22/46] 10.222.0.17 () {58 vars in 1361 bytes} [Sat Aug 6 14:16:35 2022] PUT /identity/v3/domains/default/users/b7c86189492e4a4d8e62d6e2879b6ae2/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:16:36.854670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbf32-2f0c-47eb-b538-66270b7f81d1 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:36.855102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbf32-2f0c-47eb-b538-66270b7f81d1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:36.855267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbf32-2f0c-47eb-b538-66270b7f81d1 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:36.856630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:36.861765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a977b-6c52-4c45-8ff0-bbcabf935127 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:36.862133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a977b-6c52-4c45-8ff0-bbcabf935127 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:36.862452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a977b-6c52-4c45-8ff0-bbcabf935127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:36.880731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6f6a977b-6c52-4c45-8ff0-bbcabf935127 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:36.881901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6f6a977b-6c52-4c45-8ff0-bbcabf935127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:36.891841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:36.948913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:36.971304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbbf32-2f0c-47eb-b538-66270b7f81d1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:36.982854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:37.070324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbbf32-2f0c-47eb-b538-66270b7f81d1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:37.102369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1b0a9-add5-44b4-b463-1a4aee000bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:37.103493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1b0a9-add5-44b4-b463-1a4aee000bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.103776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1b0a9-add5-44b4-b463-1a4aee000bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.104051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1b0a9-add5-44b4-b463-1a4aee000bb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.105208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1b0a9-add5-44b4-b463-1a4aee000bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:37.106664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1b0a9-add5-44b4-b463-1a4aee000bb1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:37.108289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 26/49] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:37.114152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a977b-6c52-4c45-8ff0-bbcabf935127 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:37.203119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb296e91-fb94-4838-810e-f36c5d53ad3e None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.203524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb296e91-fb94-4838-810e-f36c5d53ad3e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.203808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb296e91-fb94-4838-810e-f36c5d53ad3e None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.205160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 27/50] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:37.210065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090df5c-5bcd-4919-b8c4-dfa49608142d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.210411 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090df5c-5bcd-4919-b8c4-dfa49608142d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.210699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090df5c-5bcd-4919-b8c4-dfa49608142d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.210923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a977b-6c52-4c45-8ff0-bbcabf935127 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:37.228485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c090df5c-5bcd-4919-b8c4-dfa49608142d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:37.229666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c090df5c-5bcd-4919-b8c4-dfa49608142d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:37.245551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:37.246178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.246463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.246743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.247468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:37.248913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:37.264928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:37.279040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 24/51] 10.222.0.17 () {60 vars in 1140 bytes} [Sat Aug 6 14:16:37 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:37.300353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 28/52] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:37.323470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:37.431468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02c3a-3bab-47ba-8e74-bcb9943348d8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:37.468188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:37.468852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.469141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.469419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.481121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:37.486576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:37.487475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:37.488206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 25/53] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:37.493588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090df5c-5bcd-4919-b8c4-dfa49608142d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:37.501819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:37.598618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090df5c-5bcd-4919-b8c4-dfa49608142d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:37.631286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:37.631919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.632203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.632481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.633197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:37.633506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:37.634967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:37.643397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 29/54] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:37.648925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:37.705278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:37.752037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efeb56-15ad-4955-953c-f549f9e61f8e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:37.784567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:37.785214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.785498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.785783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] PATH_INFO: `/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.786979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c86189492e4a4d8e62d6e2879b6ae2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:37.788428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:37.790736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 26/55] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:37 2022] GET /identity/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:37.795901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:37.864821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:37.906110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4673de-1ef0-4aec-a890-7a53fd16497f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:37.940309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2f258-ee44-4533-9308-44704d6f7768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:37.940958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f258-ee44-4533-9308-44704d6f7768 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:37.941241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f258-ee44-4533-9308-44704d6f7768 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:37.941518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2f258-ee44-4533-9308-44704d6f7768 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:37.942664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2f258-ee44-4533-9308-44704d6f7768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:37.944161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2f258-ee44-4533-9308-44704d6f7768 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:37.945762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 30/56] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:37.951431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:38.049698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba322dfe-3686-4a11-9b2b-b3d504c33eb5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:38.071607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:38.082510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d121dc-4a29-4971-855e-7aba73ffe66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:38.083163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d121dc-4a29-4971-855e-7aba73ffe66c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:38.083449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d121dc-4a29-4971-855e-7aba73ffe66c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:38.083734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d121dc-4a29-4971-855e-7aba73ffe66c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:38.084531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d121dc-4a29-4971-855e-7aba73ffe66c None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.domain.id=default, user.id=b7c86189492e4a4d8e62d6e2879b6ae2) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:38.084840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d121dc-4a29-4971-855e-7aba73ffe66c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:38.086272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d121dc-4a29-4971-855e-7aba73ffe66c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:38.094990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 27/57] 10.222.0.17 () {56 vars in 1322 bytes} [Sat Aug 6 14:16:37 2022] GET /identity/v3/role_assignments?user.id=b7c86189492e4a4d8e62d6e2879b6ae2&scope.domain.id=default&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:39.292258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f815b7-b699-4a1b-a466-8b524d242f2b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:39.292776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f815b7-b699-4a1b-a466-8b524d242f2b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:39.292858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f815b7-b699-4a1b-a466-8b524d242f2b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:39.294166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 31/58] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:39.299510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f52a20-4f10-4eae-b257-b1d80d8f2085 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:39.299889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f52a20-4f10-4eae-b257-b1d80d8f2085 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:39.300068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f52a20-4f10-4eae-b257-b1d80d8f2085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:39.317769 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b3f52a20-4f10-4eae-b257-b1d80d8f2085 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:39.318997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b3f52a20-4f10-4eae-b257-b1d80d8f2085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:39.384385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 28/59] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:39.408600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f815b7-b699-4a1b-a466-8b524d242f2b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:39.505277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f815b7-b699-4a1b-a466-8b524d242f2b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:39.539405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2824ff-9edd-4e99-9696-944458a0595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:39.540061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2824ff-9edd-4e99-9696-944458a0595c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:39.540347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2824ff-9edd-4e99-9696-944458a0595c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:39.540629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2824ff-9edd-4e99-9696-944458a0595c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:39.541789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2824ff-9edd-4e99-9696-944458a0595c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:39.543315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2824ff-9edd-4e99-9696-944458a0595c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:39.544861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 32/60] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:39.550819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f52a20-4f10-4eae-b257-b1d80d8f2085 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:39.645999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f52a20-4f10-4eae-b257-b1d80d8f2085 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:39.662415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:39.678832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168bae2c-e1a0-4931-a7ce-bbd3851901f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:39.679478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bae2c-e1a0-4931-a7ce-bbd3851901f0 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:39.679765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bae2c-e1a0-4931-a7ce-bbd3851901f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:39.680045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bae2c-e1a0-4931-a7ce-bbd3851901f0 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:39.681235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bae2c-e1a0-4931-a7ce-bbd3851901f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:39.682712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bae2c-e1a0-4931-a7ce-bbd3851901f0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:39.708434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 29/61] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:16:39 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:16:42.701431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1874a3-2d79-428c-bbb9-529ce6de1c86 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:42.702478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1874a3-2d79-428c-bbb9-529ce6de1c86 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:42.702993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1874a3-2d79-428c-bbb9-529ce6de1c86 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:42.704574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 33/62] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:42.709903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab985da-96a5-40ae-b69d-d1bcebd07a28 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:42.710315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab985da-96a5-40ae-b69d-d1bcebd07a28 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:42.710466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab985da-96a5-40ae-b69d-d1bcebd07a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:42.736059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6ab985da-96a5-40ae-b69d-d1bcebd07a28 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:42.737534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6ab985da-96a5-40ae-b69d-d1bcebd07a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:42.831625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 30/63] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:42.885443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1874a3-2d79-428c-bbb9-529ce6de1c86 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:42.891997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:42.946556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf8236-4ae1-4720-b5bc-9bb8a85412a0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:42.947219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf8236-4ae1-4720-b5bc-9bb8a85412a0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:42.947726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf8236-4ae1-4720-b5bc-9bb8a85412a0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:42.949273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 31/64] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:42.954499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ada8-04ba-4279-9693-dc0952cd1b28 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:42.955031 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ada8-04ba-4279-9693-dc0952cd1b28 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:42.956041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ada8-04ba-4279-9693-dc0952cd1b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.002103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2959ada8-04ba-4279-9693-dc0952cd1b28 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:43.003361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2959ada8-04ba-4279-9693-dc0952cd1b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:43.050842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1874a3-2d79-428c-bbb9-529ce6de1c86 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.111155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e270170-6a6a-4a0b-9814-c4c5b54e14ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.112191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e270170-6a6a-4a0b-9814-c4c5b54e14ac None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.112670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e270170-6a6a-4a0b-9814-c4c5b54e14ac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.113140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e270170-6a6a-4a0b-9814-c4c5b54e14ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.131402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e270170-6a6a-4a0b-9814-c4c5b54e14ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8e270170-6a6a-4a0b-9814-c4c5b54e14ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:43.139708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:43.140541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:43.141302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:43.144668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 32/65] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:43.156222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 34/66] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:43.162078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ada8-04ba-4279-9693-dc0952cd1b28 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.172900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1116dc-d1e0-4565-babb-6852dbe45b8f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.173475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1116dc-d1e0-4565-babb-6852dbe45b8f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.173952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1116dc-d1e0-4565-babb-6852dbe45b8f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.175941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 35/67] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:43.182755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19829e40-3583-4f35-aeef-e4b629833aa5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.183317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19829e40-3583-4f35-aeef-e4b629833aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.183785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19829e40-3583-4f35-aeef-e4b629833aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.202480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-19829e40-3583-4f35-aeef-e4b629833aa5 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:43.204375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-19829e40-3583-4f35-aeef-e4b629833aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:43.207124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.276882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ada8-04ba-4279-9693-dc0952cd1b28 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.278195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 36/68] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:43.279598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19829e40-3583-4f35-aeef-e4b629833aa5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.283185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.329317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.330342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.330806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.331299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.332189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:43.332686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:43.334349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:43.343480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 33/69] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:43.344382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.397030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19829e40-3583-4f35-aeef-e4b629833aa5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.432909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d56d4-9e05-437c-acad-46f0ad6801a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.433557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d56d4-9e05-437c-acad-46f0ad6801a3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.433836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d56d4-9e05-437c-acad-46f0ad6801a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.434106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d56d4-9e05-437c-acad-46f0ad6801a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.444819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d56d4-9e05-437c-acad-46f0ad6801a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:43.446195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8f3d56d4-9e05-437c-acad-46f0ad6801a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:43.450363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:43.451248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:43.451987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 37/70] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:43.453033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d33403-8b81-4abd-9171-55987fe593ab None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.453323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d33403-8b81-4abd-9171-55987fe593ab None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.453603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d33403-8b81-4abd-9171-55987fe593ab None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.454905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 38/71] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:43.455601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d33403-8b81-4abd-9171-55987fe593ab None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.458973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df651a-b3a0-4fc7-9090-293c03fd136c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.492666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d3164-4ebe-4e8e-b0c5-56fdc536c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.493825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d3164-4ebe-4e8e-b0c5-56fdc536c265 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.494108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d3164-4ebe-4e8e-b0c5-56fdc536c265 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.494382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d3164-4ebe-4e8e-b0c5-56fdc536c265 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.504549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d3164-4ebe-4e8e-b0c5-56fdc536c265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b95d3164-4ebe-4e8e-b0c5-56fdc536c265 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:43.510476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:43.511484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:43.512429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:43.513203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.513270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 34/72] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:43.514595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648bd60-404d-4e63-8d22-4a2450f95604 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.514912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648bd60-404d-4e63-8d22-4a2450f95604 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.515202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648bd60-404d-4e63-8d22-4a2450f95604 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.516642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 35/73] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:43.517767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648bd60-404d-4e63-8d22-4a2450f95604 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.558844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d33403-8b81-4abd-9171-55987fe593ab None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.591640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62041-ea38-467c-92f7-84115adbcb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.592285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62041-ea38-467c-92f7-84115adbcb90 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.592573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62041-ea38-467c-92f7-84115adbcb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.592853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62041-ea38-467c-92f7-84115adbcb90 None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.605675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62041-ea38-467c-92f7-84115adbcb90 None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:43.607177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62041-ea38-467c-92f7-84115adbcb90 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:43.608852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 39/74] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:43.609979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea98a2-1b15-446f-88c3-33ab10027303 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.610265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea98a2-1b15-446f-88c3-33ab10027303 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.610546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea98a2-1b15-446f-88c3-33ab10027303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.617184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648bd60-404d-4e63-8d22-4a2450f95604 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.627725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e8ea98a2-1b15-446f-88c3-33ab10027303 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:43.628910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e8ea98a2-1b15-446f-88c3-33ab10027303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:43.650550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.651200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.651485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.651765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.652476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:43.652781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:43.654157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:43.662789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 36/75] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:43.662789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.682535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.699868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 40/76] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:43.701108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e28d7b-5e0b-4e8b-b1ec-c40856879216 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.701399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e28d7b-5e0b-4e8b-b1ec-c40856879216 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.701678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e28d7b-5e0b-4e8b-b1ec-c40856879216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.718327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e9e28d7b-5e0b-4e8b-b1ec-c40856879216 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:43.719478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e9e28d7b-5e0b-4e8b-b1ec-c40856879216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:43.754854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.773500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961e1a9-1b6f-42db-9eae-38456c214ac8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.785895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 41/77] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:43.786718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e28d7b-5e0b-4e8b-b1ec-c40856879216 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.809552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.810416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.810683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.810993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.811707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:43.812008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:43.813371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:43.824293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 37/78] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:43.824293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.887777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e28d7b-5e0b-4e8b-b1ec-c40856879216 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.903542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.921104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.921726 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.922010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.922289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.922560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc55ea-549c-4a60-8f64-62aacecfe803 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:43.956264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:43.956904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:43.957182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:43.957455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] PATH_INFO: `/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:43.958613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] RBAC: Authorizing `identity:get_user(user_id=b7c86189492e4a4d8e62d6e2879b6ae2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:43.960130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:43.961723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 38/79] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:43.963052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:43.968984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:43.995084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:43.996559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.063144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b4b1c-5b2a-474e-9347-e43f19c882ef None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.092829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 42/80] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:44.093680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.097633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.098334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.098618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.098951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.109015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:44.114331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:44.115168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:44.115910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 39/81] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:44.117095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.127047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.166423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.196247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfe391-d704-45a4-ae11-8bb4b8c830bf None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.217053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97fccf-58cf-406a-a183-18bce34db75c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.229233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.229844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.230126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.230409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.231560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.232999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.234579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 43/82] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:44.235371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.253866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.254509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.254844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.255151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.256320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.257758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.260129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 40/83] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:44.260778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.334574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.337662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76265-3e6a-41e1-b70c-da29f4ee9997 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.356607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2063bfe-bde0-4df5-a57e-f17edb93bc52 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.370536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.371181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.371464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.371741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.381537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.390336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.391002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.391283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.391559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.392714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.394151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.395833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 41/84] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:44.396561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.441569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.443089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.494570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89459287-3197-4c4e-a8ea-daa82de92280 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.526695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.527347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.527632 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.527907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.528610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:44.528914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.530274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.536988 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 44/85] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:44.537794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.538003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 42/86] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:44.543095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.552193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.581074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.637072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e0bf0-ce11-45af-9f9e-a0e6bc61b074 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.641004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df083b-618e-4650-b488-f2a64895fa43 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.670387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.671035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.671319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.671592 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.672263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.673492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.673694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.674111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.674647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.674959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.700611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 45/87] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:16:44 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:44.701426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.756161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.757591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.766131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.806882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:44.813683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539d8f9-73f8-4c40-9eda-6fca99b87164 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:44.858379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:44.859062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:44.859346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:44.859629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:44.860798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=303ba9e42896448ca649a81fa5154966) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:44.861386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:44.863003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:44.870996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 46/88] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/role_assignments?user.id=303ba9e42896448ca649a81fa5154966&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:44.871767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:44.879092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 43/89] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:44.879926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:45.025599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf58c6-77f0-4c25-965f-e7b392ac555f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:45.029630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9e7ad-3b6e-4d76-aef5-16b123a4d6b5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:45.034334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:45.070526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:45.072083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:45.072370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:45.072645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:45.073950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:45.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:45.077119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 44/90] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:45.077310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:45.077969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:45.078523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:45.079359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:45.079836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:45.081855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:45.084473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:45.086354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 47/91] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:44 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:45.090905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:45.127142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:45.213237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0005-de65-4606-b38e-5f4c4383bdd4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:45.261693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:45.262357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:45.262641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:45.262935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:45.267546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:45.269008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:45.275967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 45/92] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:45 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:45.276749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:45.283346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8b837-9cbf-47e2-8e42-3c57ec151d50 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:45.322778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:45.323439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:45.323725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:45.324004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:45.372598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:45.399135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:45.401000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:45.430020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:45.464075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606536fd-e01b-437e-b688-39bf3deae940 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:45.528379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:45.528932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:45.529213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:45.529490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:45.530281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=b7c86189492e4a4d8e62d6e2879b6ae2) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:45.530592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:45.534898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:45.542086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:45 2022] GET /identity/v3/role_assignments?user.id=b7c86189492e4a4d8e62d6e2879b6ae2&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:45.542699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:45.600428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 48/94] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:45.607206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:45.759014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef4d3e-040a-4a8a-8134-4c3cdef4ab4a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:45.778937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:45.824086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96ee43-9884-4f5a-9ca0-4bd3ed2ba048 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:45.829481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:45.830119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:45.830400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:45.830674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:45.831506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] RBAC: Adding query filter params (role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=303ba9e42896448ca649a81fa5154966) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:45.831817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:45.838538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:45.846286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 47/95] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:45 2022] GET /identity/v3/role_assignments?user.id=303ba9e42896448ca649a81fa5154966&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:45.894662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:45.900204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0a83-6206-4776-a311-f30b0183839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:45.900893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0a83-6206-4776-a311-f30b0183839a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:45.901178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0a83-6206-4776-a311-f30b0183839a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:45.901699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0a83-6206-4776-a311-f30b0183839a None admin] PATH_INFO: `/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:45.903245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0a83-6206-4776-a311-f30b0183839a None admin] RBAC: Authorizing `identity:get_project(project_id=bc487b05e0eb4e3ca128b4574a458b4d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:45.905159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0a83-6206-4776-a311-f30b0183839a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:45.908059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:45 2022] GET /identity/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d => generated 312 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:45.913045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:46.033858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347fad3-fc7d-4aa6-ba5c-b1d7339b065d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:46.074596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbe7b5-d50d-4dca-9afa-cbfd45f43fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:46.075278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe7b5-d50d-4dca-9afa-cbfd45f43fed None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:46.075559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe7b5-d50d-4dca-9afa-cbfd45f43fed None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:46.075842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe7b5-d50d-4dca-9afa-cbfd45f43fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:46.076634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe7b5-d50d-4dca-9afa-cbfd45f43fed None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=bc487b05e0eb4e3ca128b4574a458b4d, user.id=303ba9e42896448ca649a81fa5154966) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:46.076945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe7b5-d50d-4dca-9afa-cbfd45f43fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:46.078386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe7b5-d50d-4dca-9afa-cbfd45f43fed None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:46.084860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:45 2022] GET /identity/v3/role_assignments?user.id=303ba9e42896448ca649a81fa5154966&scope.project.id=bc487b05e0eb4e3ca128b4574a458b4d&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:47.479900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190031e1-7294-4554-b96f-da4d1a8e92bb None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:47.480444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190031e1-7294-4554-b96f-da4d1a8e92bb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:47.480444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190031e1-7294-4554-b96f-da4d1a8e92bb None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:47.481973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:47.487295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d1d5b-23c1-46d7-b903-28d3f39e6f0e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:47.487677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d1d5b-23c1-46d7-b903-28d3f39e6f0e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:47.487866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d1d5b-23c1-46d7-b903-28d3f39e6f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:47.509571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-764d1d5b-23c1-46d7-b903-28d3f39e6f0e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:47.510759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-764d1d5b-23c1-46d7-b903-28d3f39e6f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:47.542959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:47.597841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:47.629526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190031e1-7294-4554-b96f-da4d1a8e92bb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:47.786384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190031e1-7294-4554-b96f-da4d1a8e92bb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:47.834016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e8192-4a39-4016-938c-95ae60238e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:47.835070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e8192-4a39-4016-938c-95ae60238e51 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:47.835553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e8192-4a39-4016-938c-95ae60238e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:47.836024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e8192-4a39-4016-938c-95ae60238e51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:47.837940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e8192-4a39-4016-938c-95ae60238e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:47.839632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e8192-4a39-4016-938c-95ae60238e51 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:47.841520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 51/100] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:47.854903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d1d5b-23c1-46d7-b903-28d3f39e6f0e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:47.896775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820820-ced0-47d6-94c7-2ce21eff2573 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:47.897390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820820-ced0-47d6-94c7-2ce21eff2573 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:47.897884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820820-ced0-47d6-94c7-2ce21eff2573 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:47.900163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 52/101] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:47.905228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2a10e-6b7b-4245-bb79-f4be6903b615 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:47.905755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2a10e-6b7b-4245-bb79-f4be6903b615 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:47.906230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2a10e-6b7b-4245-bb79-f4be6903b615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:47.925305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-32f2a10e-6b7b-4245-bb79-f4be6903b615 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:47.926810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-32f2a10e-6b7b-4245-bb79-f4be6903b615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:48.011616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d1d5b-23c1-46d7-b903-28d3f39e6f0e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:48.031583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:48.037577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 53/102] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:48.039258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc1a42-7f8c-4d09-8b7c-10dde74d0d7d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.042892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc1a42-7f8c-4d09-8b7c-10dde74d0d7d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.042892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc1a42-7f8c-4d09-8b7c-10dde74d0d7d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.042892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 54/103] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:48.048831 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e8b05-0982-4973-ae31-8c60f5cb792b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.049574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e8b05-0982-4973-ae31-8c60f5cb792b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.049961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e8b05-0982-4973-ae31-8c60f5cb792b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.082305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-532e8b05-0982-4973-ae31-8c60f5cb792b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:48.086266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-532e8b05-0982-4973-ae31-8c60f5cb792b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:48.099576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:48.100213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.100500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.100782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.102848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:48.105082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:48.157323 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 50/104] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:16:47 2022] POST /identity/v3/projects => generated 302 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:48.158180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:48.216758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:48.244877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 55/105] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:48.275546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08d604-3fa9-4bea-a0cd-7e97fe5f038f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.275868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08d604-3fa9-4bea-a0cd-7e97fe5f038f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.276098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08d604-3fa9-4bea-a0cd-7e97fe5f038f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.277453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 56/106] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:16:48.288514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923e568-de53-49a1-8f5d-7af9e315d101 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.288789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923e568-de53-49a1-8f5d-7af9e315d101 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.288985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923e568-de53-49a1-8f5d-7af9e315d101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.346819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4923e568-de53-49a1-8f5d-7af9e315d101 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:48.347198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933c2e5-c6ba-4ca2-be9a-d8ee8da72973 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:48.348088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4923e568-de53-49a1-8f5d-7af9e315d101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:48.390376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:48.391828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.392125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.392408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.418455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:48.429649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:48.430591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:48.431387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 51/107] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:48.432008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:48.477661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 57/108] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:48.478562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923e568-de53-49a1-8f5d-7af9e315d101 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:48.529799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:48.547412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:48.560316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049a36d-46c6-47f3-9e98-ad5d8776b91e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:48.599508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923e568-de53-49a1-8f5d-7af9e315d101 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:48.605620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:48.606479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.606892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.607293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.623114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:48.631793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:48.632661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:48.633395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:48.634602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 52/109] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:48.635438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:48.667620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:48.668270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.668560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.668832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.669557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:48.669870 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:48.671291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:48.697026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 58/110] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:48.697845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:48.785994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee050548-71e7-4bea-b4cc-216ccafd32f2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:48.855091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:48.856062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.858193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.858669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.867187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:48.880053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:48.893462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:48.894326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:48.895105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 53/111] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:48.905583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d69e58-9cab-4beb-a128-00eb2b9b5ef0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:48.907665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:48.924419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:48.959327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:48.960285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:48.960568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:48.960843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:48.961583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:48.961920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:48.963328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:48.972062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 59/112] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:48.972923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.041212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78d96-30bd-47aa-90ff-9d10a54c761b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.082240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa03a8-8e01-4ba9-a619-82ef75ae7048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.083083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa03a8-8e01-4ba9-a619-82ef75ae7048 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.083374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa03a8-8e01-4ba9-a619-82ef75ae7048 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.083655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa03a8-8e01-4ba9-a619-82ef75ae7048 None admin] PATH_INFO: `/v3/users/515c859dfab34f5982478169a915d7c2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.093808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad07980-0505-47a6-83f8-78ec1034cd2e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.099370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa03a8-8e01-4ba9-a619-82ef75ae7048 None admin] RBAC: Authorizing `identity:get_user(user_id=515c859dfab34f5982478169a915d7c2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.100832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa03a8-8e01-4ba9-a619-82ef75ae7048 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.102508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 54/113] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:48 2022] GET /identity/v3/users/515c859dfab34f5982478169a915d7c2 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:49.102508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a9783-037a-4e61-8030-4a08b691fd4b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.102508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a9783-037a-4e61-8030-4a08b691fd4b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.102508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a9783-037a-4e61-8030-4a08b691fd4b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.102508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 55/114] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:49.102508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a9783-037a-4e61-8030-4a08b691fd4b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.135697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09963ad-6db8-4bd3-9ca7-ad1e2959f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.136749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09963ad-6db8-4bd3-9ca7-ad1e2959f3ad None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.137505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09963ad-6db8-4bd3-9ca7-ad1e2959f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.138241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09963ad-6db8-4bd3-9ca7-ad1e2959f3ad None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.139296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09963ad-6db8-4bd3-9ca7-ad1e2959f3ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:49.139773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09963ad-6db8-4bd3-9ca7-ad1e2959f3ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.141853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09963ad-6db8-4bd3-9ca7-ad1e2959f3ad None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.152696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 60/115] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:49.154449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16fd30-5bb3-418a-bef7-bec5e7ad28a7 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.155151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16fd30-5bb3-418a-bef7-bec5e7ad28a7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.155572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16fd30-5bb3-418a-bef7-bec5e7ad28a7 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.157666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 61/116] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:49.158838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16fd30-5bb3-418a-bef7-bec5e7ad28a7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.167111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.208874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.220396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a9783-037a-4e61-8030-4a08b691fd4b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.260053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ece12-ff9e-4be3-906f-32931d94ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.260703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ece12-ff9e-4be3-906f-32931d94ec0b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.260985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ece12-ff9e-4be3-906f-32931d94ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.261262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ece12-ff9e-4be3-906f-32931d94ec0b None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.262442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ece12-ff9e-4be3-906f-32931d94ec0b None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.263912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ece12-ff9e-4be3-906f-32931d94ec0b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.265504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 56/117] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:49.266608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef210-8c61-47f2-b268-2048dad1af56 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.266904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef210-8c61-47f2-b268-2048dad1af56 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.267175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef210-8c61-47f2-b268-2048dad1af56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.284126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16fd30-5bb3-418a-bef7-bec5e7ad28a7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.285295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4b4ef210-8c61-47f2-b268-2048dad1af56 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:49.286468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4b4ef210-8c61-47f2-b268-2048dad1af56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:49.320857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.322083 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.322365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.322637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.356217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 57/118] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:49.357235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef210-8c61-47f2-b268-2048dad1af56 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.404470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.405869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.414531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.448832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.466468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef210-8c61-47f2-b268-2048dad1af56 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.502676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a74702-f4e6-461e-bbfb-6137bfd32c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.503730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a74702-f4e6-461e-bbfb-6137bfd32c4a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.504119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a74702-f4e6-461e-bbfb-6137bfd32c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.504395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a74702-f4e6-461e-bbfb-6137bfd32c4a None admin] PATH_INFO: `/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.505600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a74702-f4e6-461e-bbfb-6137bfd32c4a None admin] RBAC: Authorizing `identity:get_user(user_id=b7c86189492e4a4d8e62d6e2879b6ae2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.507064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a74702-f4e6-461e-bbfb-6137bfd32c4a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.509467 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 58/119] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:49.510456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbf38e-7f38-4dfd-9d6a-c5c8a52a1d4f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.510762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbf38e-7f38-4dfd-9d6a-c5c8a52a1d4f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.511041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbf38e-7f38-4dfd-9d6a-c5c8a52a1d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.522653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 62/120] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:49.527877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.538413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a6fbf38e-7f38-4dfd-9d6a-c5c8a52a1d4f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:49.540704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a6fbf38e-7f38-4dfd-9d6a-c5c8a52a1d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:49.620535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 59/121] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:49.621397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbf38e-7f38-4dfd-9d6a-c5c8a52a1d4f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.639733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ed46-ecd1-4fa7-99d7-c10dfabf219f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.678962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.679536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.679814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.680090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.681908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.683400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.684729 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.689959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 63/122] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:49.689959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.697005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.744891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbf38e-7f38-4dfd-9d6a-c5c8a52a1d4f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.780779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1203-d03c-4660-a822-4946624a935f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.781427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1203-d03c-4660-a822-4946624a935f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.781708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1203-d03c-4660-a822-4946624a935f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.781982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1203-d03c-4660-a822-4946624a935f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.792771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f1203-d03c-4660-a822-4946624a935f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:49.793034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45ac0-9dd1-46a3-9dc9-365783355504 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-756f1203-d03c-4660-a822-4946624a935f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:49.797631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:49.798484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 60/123] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:49.803591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1203-d03c-4660-a822-4946624a935f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.834807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.835483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.835797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.836077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.837274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.838724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.840437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 64/124] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:49.841207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.922354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1203-d03c-4660-a822-4946624a935f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.929433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.936032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:49.955128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f56b49-1af5-4a4f-9dc7-622fb3486009 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:49.962713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.963451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.963740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.964488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:49.966120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:49.967860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:49.970759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 61/125] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:49.971320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:49.992526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:49.993964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:49.994250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:49.994530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.008435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:50.014025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:50.014859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:50.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 65/126] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:16:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:50.016513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.082432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab7f25-deac-4600-9d32-3deac5252c63 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.119038 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.119767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.120023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa37f2-e272-4a25-adb8-cd0be7ca50ec None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.120122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.120428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/303ba9e42896448ca649a81fa5154966/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.122402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] RBAC: Authorizing `identity:create_grant(project_id=6563532b47244425a9fbdda6e507bf2c, user_id=303ba9e42896448ca649a81fa5154966, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.123879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.138037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 62/127] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:16:49 2022] PUT /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/303ba9e42896448ca649a81fa5154966/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:16:50.141713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.166699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.167447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.167731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.168010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.168914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:50.169272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.171121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.174010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:50.184301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 66/128] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:50.185089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.200049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:50.285925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60399942-b552-4045-89c6-e9fd97217938 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.316912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156e98-2d3d-493b-919a-08679ae5edfd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.331810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.333087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.333806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.334078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/b7c86189492e4a4d8e62d6e2879b6ae2/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.336540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] RBAC: Authorizing `identity:create_grant(project_id=6563532b47244425a9fbdda6e507bf2c, user_id=b7c86189492e4a4d8e62d6e2879b6ae2, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.338515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.356997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.357722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.358007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.358283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.358382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 63/129] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:16:50 2022] PUT /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/b7c86189492e4a4d8e62d6e2879b6ae2/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:16:50.359169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] RBAC: Adding query filter params (role.id=135b77755ad64d97bfb82803f1c574f6, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=515c859dfab34f5982478169a915d7c2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:50.359294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.359570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.364993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.370820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 67/130] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:50 2022] GET /identity/v3/role_assignments?user.id=515c859dfab34f5982478169a915d7c2&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=135b77755ad64d97bfb82803f1c574f6 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:50.371771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.470591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:50.512754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:50.517702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082199-efef-4a19-a664-8b63fed4aeda None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.548384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d740e1-db1c-4385-be73-dc41b74858f5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.602911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.603578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.603842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.604123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.604863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:50.605172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.607043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.619012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 64/131] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:16:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:50.625704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.645826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a8f3e-3960-44a1-9e0b-26f3f4dec85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.646510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8f3e-3960-44a1-9e0b-26f3f4dec85e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.647814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8f3e-3960-44a1-9e0b-26f3f4dec85e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.648106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8f3e-3960-44a1-9e0b-26f3f4dec85e None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.650890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a8f3e-3960-44a1-9e0b-26f3f4dec85e None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.652408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a8f3e-3960-44a1-9e0b-26f3f4dec85e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.654091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 68/132] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:50 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:50.655406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7374317-2cc9-4824-9bbd-83a4b3d39aa5 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.655695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7374317-2cc9-4824-9bbd-83a4b3d39aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.655976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7374317-2cc9-4824-9bbd-83a4b3d39aa5 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.657277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 69/133] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:50.662763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7374317-2cc9-4824-9bbd-83a4b3d39aa5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.761413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916fcfa-320b-42d9-864d-0a7548a08984 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.813156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7374317-2cc9-4824-9bbd-83a4b3d39aa5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:50.830319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c73348-7743-46d1-b487-6eb4e3aea01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.831324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c73348-7743-46d1-b487-6eb4e3aea01b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.831807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c73348-7743-46d1-b487-6eb4e3aea01b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.832276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c73348-7743-46d1-b487-6eb4e3aea01b None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.833693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c73348-7743-46d1-b487-6eb4e3aea01b None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.835495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c73348-7743-46d1-b487-6eb4e3aea01b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.846905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 65/134] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:50 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:50.847076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:50.850303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa549-2f02-4618-9084-71cac566f9f6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.850303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa549-2f02-4618-9084-71cac566f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.850303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa549-2f02-4618-9084-71cac566f9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.850668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:50.853089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:50.853089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:50.853089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] PATH_INFO: `/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:50.854901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=bc487b05e0eb4e3ca128b4574a458b4d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:50.855339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:50.857467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 70/135] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:50 2022] GET /identity/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:50.859022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:50.884195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-79ffa549-2f02-4618-9084-71cac566f9f6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:50.885405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-79ffa549-2f02-4618-9084-71cac566f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:50.949576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:50.965967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa26745-2160-49da-a708-e33aa7c3e2a2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:51.001665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:51.002318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:51.002600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:51.002891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:51.004078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:51.005528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:51.007292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 71/136] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:50 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:51.008075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:51.056208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 66/137] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:51.057149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa549-2f02-4618-9084-71cac566f9f6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:51.090959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:51.138347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb89dc-47f5-443d-8eec-e2144ccd9079 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:51.173663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:51.174645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:51.174996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:51.175275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] PATH_INFO: `/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d/users/303ba9e42896448ca649a81fa5154966/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:51.177254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc487b05e0eb4e3ca128b4574a458b4d, user_id=303ba9e42896448ca649a81fa5154966, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:51.178816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:51.200293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 72/138] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:16:51 2022] PUT /identity/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d/users/303ba9e42896448ca649a81fa5154966/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:16:51.202444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:51.314151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa549-2f02-4618-9084-71cac566f9f6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:51.381638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:51.382335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:51.382620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:51.382915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/303ba9e42896448ca649a81fa5154966/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:51.388153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] RBAC: Authorizing `identity:create_grant(project_id=6563532b47244425a9fbdda6e507bf2c, user_id=303ba9e42896448ca649a81fa5154966, role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:51.391782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:51.423394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 67/139] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:16:51 2022] PUT /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/303ba9e42896448ca649a81fa5154966/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:16:51.425976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba3e1-5084-42d8-a441-9144c9538fe8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:51.459644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:51.466345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9553ab-aa28-4570-a1b4-5d138ac26d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:51.467073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9553ab-aa28-4570-a1b4-5d138ac26d9f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:51.467431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9553ab-aa28-4570-a1b4-5d138ac26d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:51.467681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9553ab-aa28-4570-a1b4-5d138ac26d9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:51.469042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9553ab-aa28-4570-a1b4-5d138ac26d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:51.471051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9553ab-aa28-4570-a1b4-5d138ac26d9f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:51.476460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 73/140] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:51.479111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:51.500798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:51.602155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182740-9f7a-41d3-ae60-e5894d292127 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:51.642106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac46231-52e3-4d91-bfb5-6c9ec9253031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:51.642891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac46231-52e3-4d91-bfb5-6c9ec9253031 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:51.643129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac46231-52e3-4d91-bfb5-6c9ec9253031 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:51.643404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac46231-52e3-4d91-bfb5-6c9ec9253031 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:51.644122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac46231-52e3-4d91-bfb5-6c9ec9253031 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:51.645560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac46231-52e3-4d91-bfb5-6c9ec9253031 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:51.671835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 68/141] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:16:51 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:52.720808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9b049-b177-4279-b00c-4bbe03c150dd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:52.721312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9b049-b177-4279-b00c-4bbe03c150dd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:52.721386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9b049-b177-4279-b00c-4bbe03c150dd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:52.722825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 74/142] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:52.729411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fea0a4-7bf1-4d8d-b223-eeee92d66ad0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:52.729827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fea0a4-7bf1-4d8d-b223-eeee92d66ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:52.729996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fea0a4-7bf1-4d8d-b223-eeee92d66ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:52.752187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d8fea0a4-7bf1-4d8d-b223-eeee92d66ad0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:52.753387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d8fea0a4-7bf1-4d8d-b223-eeee92d66ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:52.830938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 69/143] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:52.854914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9b049-b177-4279-b00c-4bbe03c150dd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:52.962130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9b049-b177-4279-b00c-4bbe03c150dd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:52.999095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd5b8d-05e8-4065-b62c-f8eb8ac9c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:52.999658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd5b8d-05e8-4065-b62c-f8eb8ac9c9ba None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:52.999938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd5b8d-05e8-4065-b62c-f8eb8ac9c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.000213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd5b8d-05e8-4065-b62c-f8eb8ac9c9ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.010903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd5b8d-05e8-4065-b62c-f8eb8ac9c9ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-64cd5b8d-05e8-4065-b62c-f8eb8ac9c9ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:53.015828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:53.016675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:53.017428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 75/144] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:53.023284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fea0a4-7bf1-4d8d-b223-eeee92d66ad0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:53.026997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:53.041115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f88b-893d-46de-b91b-c965857fa64f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.041441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f88b-893d-46de-b91b-c965857fa64f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.041725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f88b-893d-46de-b91b-c965857fa64f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.044212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 76/145] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:53.048381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7443-9747-4bd6-8a20-727d9151e9e9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.048660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7443-9747-4bd6-8a20-727d9151e9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.048930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7443-9747-4bd6-8a20-727d9151e9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.080511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7f7f7443-9747-4bd6-8a20-727d9151e9e9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:53.081707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7f7f7443-9747-4bd6-8a20-727d9151e9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:53.140800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:53.161099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 77/146] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:53.185037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f7443-9747-4bd6-8a20-727d9151e9e9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:53.204940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fea0a4-7bf1-4d8d-b223-eeee92d66ad0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:53.264281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:53.265335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.265819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.266346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.267310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:53.267815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:53.269626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:53.285344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 70/147] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:53.291698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f7443-9747-4bd6-8a20-727d9151e9e9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:53.310702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:53.337156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:53.340127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.340127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.340127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.349592 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:53.355298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:53.356729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:53.357530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 78/148] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:16:53.365769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:53.414442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:53.428456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:53.472082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22018d6-80f7-4b88-8cdc-16fbdb6cda67 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:53.486994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad30d57-4a9d-4e2d-868a-2db3fa56916e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:53.506836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:53.507498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.507775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.508048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.509443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:53.509637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:53.511060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:53.520066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 79/149] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:53.525967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:53.552970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:53.554057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.554549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.555107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.557574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:53.559313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:53.561256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 71/150] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:53 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:53.566799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:53.630181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586ffe9-df28-48cb-ab2d-ec70b6d6ab6d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:53.636847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:53.669699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620877c-23f3-4fe8-97f5-9b0af31116d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:53.670355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620877c-23f3-4fe8-97f5-9b0af31116d4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.670633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620877c-23f3-4fe8-97f5-9b0af31116d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.670927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620877c-23f3-4fe8-97f5-9b0af31116d4 None admin] PATH_INFO: `/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.672109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620877c-23f3-4fe8-97f5-9b0af31116d4 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c86189492e4a4d8e62d6e2879b6ae2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:53.673626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620877c-23f3-4fe8-97f5-9b0af31116d4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:53.675816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 80/151] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:53 2022] GET /identity/v3/users/b7c86189492e4a4d8e62d6e2879b6ae2 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:53.676943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109a215-933f-46a5-943a-c824ccac721d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.677220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109a215-933f-46a5-943a-c824ccac721d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.677491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109a215-933f-46a5-943a-c824ccac721d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.678692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 81/152] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:53.684189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266bd38-ffe8-4dde-877b-2111524360c7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.684458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266bd38-ffe8-4dde-877b-2111524360c7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.684731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266bd38-ffe8-4dde-877b-2111524360c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.703871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3266bd38-ffe8-4dde-877b-2111524360c7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:53.705366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3266bd38-ffe8-4dde-877b-2111524360c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:53.751946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb9012-5c31-4958-810f-583f2d049fb9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:53.773989 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 82/153] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:53.775538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266bd38-ffe8-4dde-877b-2111524360c7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:53.812833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:53.813711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.814139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.814560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.818972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:53.879138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266bd38-ffe8-4dde-877b-2111524360c7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:53.884270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:53.916586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:53.917373 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:53.917750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:53.918123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:53.972470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:53.973925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:54.045677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:54.047120 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:54.163706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 72/154] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 14:16:54.164563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:54.229525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 83/155] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:54.230380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:54.239204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:54.241435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:54.350505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30625602-7d99-4306-94f9-5e06918de2f7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:54.420508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:54.421160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:54.422447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:54.422729 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:54.427835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497d48c-c0cb-4e04-a78b-ff5df4129e5b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:54.453776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:54.458742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:54.459745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:54.460655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 73/156] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:54.465359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:54.472435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:54.473098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:54.473374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:54.473644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:54.474818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:54.476314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:54.478709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 84/157] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:54 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:54.479419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:54.586161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:54.635140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb7da-0a2e-459a-8cc1-c5322d92eda4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:54.643129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:54.672829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883acd9b-124d-4c15-b505-6aa0e582276a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:54.693683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:54.694675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:54.695304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:54.695756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:54.699726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:54.701906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:54.704262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 74/158] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:54 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:54.705329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:54.753923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:54.754563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:54.754837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:54.755128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:54.756043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:54.756517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:54.758076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:54.772931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 85/159] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:54.773965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:54.847657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c162964-9edc-4b15-8330-7d3fce51bf79 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:54.888569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:54.889857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:54.890284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:54.890700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:54.891981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=303ba9e42896448ca649a81fa5154966) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:54.892434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:54.894271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:54.901476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 75/160] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:54 2022] GET /identity/v3/role_assignments?user.id=303ba9e42896448ca649a81fa5154966&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:54.902458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:54.937230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:55.025475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e0b1-24ad-4356-9d41-a3adab2acfb8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:55.055354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0e529-b52b-48e6-bb2a-d5111ea74198 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:55.072586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47b100-85c2-4549-8b71-3855872d913c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:55.073516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b100-85c2-4549-8b71-3855872d913c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.073967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b100-85c2-4549-8b71-3855872d913c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.074397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b100-85c2-4549-8b71-3855872d913c None admin] PATH_INFO: `/v3/users/515c859dfab34f5982478169a915d7c2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.079723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47b100-85c2-4549-8b71-3855872d913c None admin] RBAC: Authorizing `identity:get_user(user_id=515c859dfab34f5982478169a915d7c2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:55.081897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47b100-85c2-4549-8b71-3855872d913c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:55.084099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 76/161] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:54 2022] GET /identity/v3/users/515c859dfab34f5982478169a915d7c2 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:55.085580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34453-880d-4cf4-aa02-fbe5e23ffe94 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.086026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34453-880d-4cf4-aa02-fbe5e23ffe94 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.086445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34453-880d-4cf4-aa02-fbe5e23ffe94 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.088797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 77/162] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:55.094327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5606-bc11-4ac3-9713-c93d21524294 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.094796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5606-bc11-4ac3-9713-c93d21524294 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.095231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5606-bc11-4ac3-9713-c93d21524294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.128663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5b4f5606-bc11-4ac3-9713-c93d21524294 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:55.130538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5b4f5606-bc11-4ac3-9713-c93d21524294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:55.145396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:55.146319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.146781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.147242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.148201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=b7c86189492e4a4d8e62d6e2879b6ae2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:55.148692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:55.150925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:55.153061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:55.177258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 86/163] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:54 2022] GET /identity/v3/role_assignments?user.id=b7c86189492e4a4d8e62d6e2879b6ae2&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:55.178424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:55.241996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 78/164] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:55.304992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5606-bc11-4ac3-9713-c93d21524294 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:55.338996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:55.416978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7874f6-d497-462d-8760-a94a741f2d45 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:55.445923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5606-bc11-4ac3-9713-c93d21524294 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:55.487456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:55.488994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.489446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.490276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.500913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:55.501584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.501849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.502134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.504024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:55.506781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:55.508288 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:55.510151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:55.511147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:16:55.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:55.513061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 87/165] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:55 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:55.514415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 79/166] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:55.520321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:55.520756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:55.623619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:55.645167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772c01d-2daa-4bfd-a1c6-d1d7531df38a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:55.648406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:55.689961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:55.690555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.690989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.691275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/515c859dfab34f5982478169a915d7c2/roles/135b77755ad64d97bfb82803f1c574f6` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.693872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6563532b47244425a9fbdda6e507bf2c, user_id=515c859dfab34f5982478169a915d7c2, role_id=135b77755ad64d97bfb82803f1c574f6)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:55.695340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:55.714645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 80/167] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:16:55 2022] PUT /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c/users/515c859dfab34f5982478169a915d7c2/roles/135b77755ad64d97bfb82803f1c574f6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:16:55.720378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa894e7-aca9-49f9-8dbe-57c72226cecc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:55.784872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041beea8-ffd9-4ada-b370-6783c063c46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:55.785804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041beea8-ffd9-4ada-b370-6783c063c46b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.786252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041beea8-ffd9-4ada-b370-6783c063c46b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.786679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041beea8-ffd9-4ada-b370-6783c063c46b None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.787599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041beea8-ffd9-4ada-b370-6783c063c46b None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:55.788056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041beea8-ffd9-4ada-b370-6783c063c46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:55.790191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041beea8-ffd9-4ada-b370-6783c063c46b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:55.803413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 88/168] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:55.810091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:55.943113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc1b33-f5c8-485f-8126-31715fd484d4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:55.955885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3502dc5-9dba-4451-bca4-9cfcf954776d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.956203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3502dc5-9dba-4451-bca4-9cfcf954776d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.956451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3502dc5-9dba-4451-bca4-9cfcf954776d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.958032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 89/169] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:55.964344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b70f-aff1-4e35-80d5-4f00e382ac39 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.964926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b70f-aff1-4e35-80d5-4f00e382ac39 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.965209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b70f-aff1-4e35-80d5-4f00e382ac39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.983352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:55.984128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:55.984410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:55.984734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:55.985919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:55.987396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:55.990718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 81/170] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:55 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:55.996165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:56.011889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-37d0b70f-aff1-4e35-80d5-4f00e382ac39 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:56.013096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-37d0b70f-aff1-4e35-80d5-4f00e382ac39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:56.013488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:56.123187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 90/171] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:16:56.126995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580e581-f2f4-4e9c-8db3-9c5951056d80 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:56.150398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b70f-aff1-4e35-80d5-4f00e382ac39 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:56.164670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:56.166333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.166602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.166948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.226840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:56.265245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:56.267086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:56.272492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b70f-aff1-4e35-80d5-4f00e382ac39 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:56.284721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:56.310718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:56.311902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.312393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.312873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.324039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:56.329944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:56.330810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:56.331717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:56.334210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 91/172] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:16:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:16:56.341154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:56.390477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 82/173] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:56.396177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:56.499003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f171361-9fb3-4f0e-bb1e-916edce591a1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:56.529413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:56.558552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:56.558963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.559245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.559516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.560263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:56.560538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:56.561914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:56.566229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb332b4-f632-47d0-9a58-4fe353a60e64 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:56.574718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 92/174] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:16:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:56.586236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:56.589310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:56.614229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:56.615355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.615903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.616392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] PATH_INFO: `/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.617826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] RBAC: Authorizing `identity:get_project(project_id=bc487b05e0eb4e3ca128b4574a458b4d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:56.619578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:56.621813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 83/175] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:56 2022] GET /identity/v3/projects/bc487b05e0eb4e3ca128b4574a458b4d => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:56.627486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:56.721349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530c524-4781-4301-9d01-f39891f8b7fb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:56.751581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e048-600e-4498-9c8c-4efd4871295d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:56.769211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c683d7-cbac-4c04-8a2d-c06a26b84b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:56.770114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c683d7-cbac-4c04-8a2d-c06a26b84b11 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.770810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c683d7-cbac-4c04-8a2d-c06a26b84b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.771670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c683d7-cbac-4c04-8a2d-c06a26b84b11 None admin] PATH_INFO: `/v3/users/303ba9e42896448ca649a81fa5154966` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.773031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c683d7-cbac-4c04-8a2d-c06a26b84b11 None admin] RBAC: Authorizing `identity:get_user(user_id=303ba9e42896448ca649a81fa5154966)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:56.775735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c683d7-cbac-4c04-8a2d-c06a26b84b11 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:56.778386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 93/176] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:56 2022] GET /identity/v3/users/303ba9e42896448ca649a81fa5154966 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:56.780368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1290e7-f843-472f-83e1-f50049082c9b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.780815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1290e7-f843-472f-83e1-f50049082c9b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.781227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1290e7-f843-472f-83e1-f50049082c9b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.786535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 94/177] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:56.788356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc92d5-66f8-48ef-b80a-14468eab5e8d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.788804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc92d5-66f8-48ef-b80a-14468eab5e8d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.789749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc92d5-66f8-48ef-b80a-14468eab5e8d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.791673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 95/178] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:56.793315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc92d5-66f8-48ef-b80a-14468eab5e8d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:56.798602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8c9bc-fa22-4235-9aa5-3897c5f2e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:56.799153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8c9bc-fa22-4235-9aa5-3897c5f2e924 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.799804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8c9bc-fa22-4235-9aa5-3897c5f2e924 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.800184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8c9bc-fa22-4235-9aa5-3897c5f2e924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.801083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8c9bc-fa22-4235-9aa5-3897c5f2e924 None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=bc487b05e0eb4e3ca128b4574a458b4d, user.id=303ba9e42896448ca649a81fa5154966) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:56.801499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8c9bc-fa22-4235-9aa5-3897c5f2e924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:56.803082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8c9bc-fa22-4235-9aa5-3897c5f2e924 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:56.810673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 84/179] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:56 2022] GET /identity/v3/role_assignments?user.id=303ba9e42896448ca649a81fa5154966&scope.project.id=bc487b05e0eb4e3ca128b4574a458b4d&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:56.816085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0918-7a6e-4be3-83cd-9cef1ad94ae4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.816085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0918-7a6e-4be3-83cd-9cef1ad94ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.816085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0918-7a6e-4be3-83cd-9cef1ad94ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:56.847723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-52af0918-7a6e-4be3-83cd-9cef1ad94ae4 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:56.855267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-52af0918-7a6e-4be3-83cd-9cef1ad94ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:56.907732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:56.939314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:56.973407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 85/180] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:56.975693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc92d5-66f8-48ef-b80a-14468eab5e8d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:56.978077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1db7-e0ef-4fb9-8bbd-e5109b4c636a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:56.978622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1db7-e0ef-4fb9-8bbd-e5109b4c636a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:56.979123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1db7-e0ef-4fb9-8bbd-e5109b4c636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.003959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9cbb1db7-e0ef-4fb9-8bbd-e5109b4c636a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:57.005012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9cbb1db7-e0ef-4fb9-8bbd-e5109b4c636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:57.029134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0c779-c3f5-4b38-a8e1-a7f78c36a6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.030254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0c779-c3f5-4b38-a8e1-a7f78c36a6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.030715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0c779-c3f5-4b38-a8e1-a7f78c36a6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.031218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0c779-c3f5-4b38-a8e1-a7f78c36a6a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.109707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 86/181] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:57.110846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed96eca-c320-4781-b3db-275543c1f4f5 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.111160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed96eca-c320-4781-b3db-275543c1f4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.111347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0c779-c3f5-4b38-a8e1-a7f78c36a6a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:57.111445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed96eca-c320-4781-b3db-275543c1f4f5 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.112794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 87/182] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:57.112905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0c779-c3f5-4b38-a8e1-a7f78c36a6a1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:57.113342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed96eca-c320-4781-b3db-275543c1f4f5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.167740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:57.226562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 96/183] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:57.226562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857c1f9-b245-451d-8e81-5aa56a5172ba None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.226562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857c1f9-b245-451d-8e81-5aa56a5172ba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.226562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857c1f9-b245-451d-8e81-5aa56a5172ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.235749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed96eca-c320-4781-b3db-275543c1f4f5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.238057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:57.243469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3857c1f9-b245-451d-8e81-5aa56a5172ba None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:57.244643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3857c1f9-b245-451d-8e81-5aa56a5172ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:57.272960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.274079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.274844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.275157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.285766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:57.291194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:57.292036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:57.292804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 88/184] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:57.295215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.316425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 97/185] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:57.327728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857c1f9-b245-451d-8e81-5aa56a5172ba None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.411301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676aaf0-2ec6-4861-8d37-b0a5e228c9ae None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.416769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:57.441859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857c1f9-b245-451d-8e81-5aa56a5172ba None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.456344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.457073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.457355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.457646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.469734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:57.475238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:57.476113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:16:57.476881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 89/186] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:57.478202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.484935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.485584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.485861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.486137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.487316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:57.488770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:57.490426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 98/187] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:57.495814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.505397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:57.592473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd536b-0901-4773-8c8b-baedddf862bd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.600912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d622210-ef0a-48a3-8d9e-d40ef5c27624 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.628589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.629269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.629487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.629762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.630592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:57.630779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:57.632208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:57.636160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.636805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.637082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.637353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.638512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:57.639994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:57.641117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 90/188] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:57.641572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 99/189] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:57.642070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.642322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.658299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:57.745731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:57.753009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5589ee-f30b-481e-b55c-e11ae00854a8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.763128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa51c3-ef6e-4a67-ae3e-52a1fe287a85 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.787777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.788399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.788678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.788959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.789766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] RBAC: Adding query filter params (role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=303ba9e42896448ca649a81fa5154966) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:57.790076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:57.791554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:57.806903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 100/190] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/role_assignments?user.id=303ba9e42896448ca649a81fa5154966&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:57.806903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.808378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.809106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.809453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.809798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:57.810744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:57.811159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:57.812747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:57.823226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 91/191] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:57.824186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:57.928889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:57.945034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0ec46-df17-4fae-887d-12f0ba10a080 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.963324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622e545-b2ad-4a96-a130-62589cf53b40 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:57.983144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:57.983798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:57.984077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:57.984351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] PATH_INFO: `/v3/users/82d41fd784e54d0ab1421c2da44dff16` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.003378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] RBAC: Authorizing `identity:get_user(user_id=82d41fd784e54d0ab1421c2da44dff16)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.004851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.006837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 92/192] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:57 2022] GET /identity/v3/users/82d41fd784e54d0ab1421c2da44dff16 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:58.011059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:58.011932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.012422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.012884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.012995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:58.013872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.015380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.025133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:58.053271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 101/193] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:16:57 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:16:58.058915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:58.161166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65a60-6151-4976-beed-9db7127809dc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:58.201271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b57d7-9e82-4940-89a2-cc7e331602aa None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:58.213770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:58.214556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.214843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.218347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] PATH_INFO: `/v3/users/82d41fd784e54d0ab1421c2da44dff16` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.220685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] RBAC: Authorizing `identity:get_user(user_id=82d41fd784e54d0ab1421c2da44dff16)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.222329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.223502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 93/194] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/users/82d41fd784e54d0ab1421c2da44dff16 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:58.229111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:58.245242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783534b9-4da9-49c6-9d63-5de17394cec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:58.246063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783534b9-4da9-49c6-9d63-5de17394cec8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.246351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783534b9-4da9-49c6-9d63-5de17394cec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.246623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783534b9-4da9-49c6-9d63-5de17394cec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.250349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:58.330276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:58.352844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783534b9-4da9-49c6-9d63-5de17394cec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.354838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783534b9-4da9-49c6-9d63-5de17394cec8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.362176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012244c-fb2d-4bea-8573-459731254859 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:58.408485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb430b4-66ee-4f4e-b958-73542b933342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:58.409256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb430b4-66ee-4f4e-b958-73542b933342 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.409467 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb430b4-66ee-4f4e-b958-73542b933342 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.409728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb430b4-66ee-4f4e-b958-73542b933342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.483995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 102/195] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:58.485709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8360-c21f-4f5a-ab2d-69c4a11cd65e None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.486192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8360-c21f-4f5a-ab2d-69c4a11cd65e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.486660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8360-c21f-4f5a-ab2d-69c4a11cd65e None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.488173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 103/196] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:16:58.490022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8360-c21f-4f5a-ab2d-69c4a11cd65e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:58.508020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb430b4-66ee-4f4e-b958-73542b933342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.510433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb430b4-66ee-4f4e-b958-73542b933342 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.529865 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:58.610327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8360-c21f-4f5a-ab2d-69c4a11cd65e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:58.613868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:58.648424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:58.649523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.650079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.650182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 94/197] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:58.650706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.651655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f2618-90f2-44e6-abd7-54e108ada548 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.651926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f2618-90f2-44e6-abd7-54e108ada548 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.652212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f2618-90f2-44e6-abd7-54e108ada548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.652920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.654546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.656946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 104/198] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:58.657987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:58.675371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cf6f2618-90f2-44e6-abd7-54e108ada548 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:16:58.677013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cf6f2618-90f2-44e6-abd7-54e108ada548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:16:58.770492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 95/199] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:16:58.771827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f2618-90f2-44e6-abd7-54e108ada548 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:58.790502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156cd39-06ae-49f5-a40d-539f50f3aaa6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:58.792320 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:58.832665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:58.833244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.833565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.833849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.835630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.838979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.840812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 105/200] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:58.842304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:58.915875 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f2618-90f2-44e6-abd7-54e108ada548 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:58.919225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:58.957291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:58.958002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:58.958286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:58.958707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:58.959524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] RBAC: Adding query filter params (role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=82d41fd784e54d0ab1421c2da44dff16) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:58.959836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:58.961333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:58.966163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5c76-30b9-41b5-9d72-11304c509253 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:58.975215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 96/201] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/role_assignments?user.id=82d41fd784e54d0ab1421c2da44dff16&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:58.976241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:59.006742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:59.007786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:59.008977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:59.009456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:59.023360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:16:59.030393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:16:59.031405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:16:59.032178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:16:59.035473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 106/202] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:16:59.045986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:59.089322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:59.197977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669c392-15fa-4c21-8442-19df7ced70e6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:59.199701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deffd5a-251a-456c-9191-f3a25e620d02 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:59.234497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:59.235254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:59.235477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:59.235758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:59.236523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:59.236829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:59.238272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:59.241119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:59.241745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:59.242029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:59.242306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:59.243169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=82d41fd784e54d0ab1421c2da44dff16) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:16:59.243951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:59.245675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:59.247048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 107/203] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:16:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:59.251926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 97/204] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:16:58 2022] GET /identity/v3/role_assignments?user.id=82d41fd784e54d0ab1421c2da44dff16&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:16:59.252498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:59.411107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:59.420969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bbeff-1701-4b48-ba24-5cc53d7eac83 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:59.460199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:59.460890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:59.461249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:59.461961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] PATH_INFO: `/v3/users/515c859dfab34f5982478169a915d7c2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:59.463631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] RBAC: Authorizing `identity:get_user(user_id=515c859dfab34f5982478169a915d7c2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:59.465943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:59.468496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 108/205] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:16:59 2022] GET /identity/v3/users/515c859dfab34f5982478169a915d7c2 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:16:59.476100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:16:59.523393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:59.614102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3c34-5bac-4026-94a2-f85950193678 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:16:59.658657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:16:59.659696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:16:59.660193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:16:59.660667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:16:59.758404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:16:59.760226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:16:59.804305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:16:59.889512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 98/206] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:16:59.898337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:00.022233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2f1b-1064-4a11-99cd-9cd8a7285e05 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:00.062301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002827e-57be-4ece-aa5a-4872e3cb5347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:00.062981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002827e-57be-4ece-aa5a-4872e3cb5347 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:00.063268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002827e-57be-4ece-aa5a-4872e3cb5347 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:00.063544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002827e-57be-4ece-aa5a-4872e3cb5347 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:00.064723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002827e-57be-4ece-aa5a-4872e3cb5347 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:00.066178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002827e-57be-4ece-aa5a-4872e3cb5347 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:00.068075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 109/207] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:16:59 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:00.073743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:00.186134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490981f7-4a7a-4b3d-b784-d91130b0af6a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:00.227447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:00.236087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30f72c-b3e6-4d67-964e-b824ee4b4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:00.236932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30f72c-b3e6-4d67-964e-b824ee4b4f9b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:00.237407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30f72c-b3e6-4d67-964e-b824ee4b4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:00.237891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30f72c-b3e6-4d67-964e-b824ee4b4f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:00.238693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30f72c-b3e6-4d67-964e-b824ee4b4f9b None admin] RBAC: Adding query filter params (role.id=135b77755ad64d97bfb82803f1c574f6, scope.project.id=6563532b47244425a9fbdda6e507bf2c, user.id=515c859dfab34f5982478169a915d7c2) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:00.239036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30f72c-b3e6-4d67-964e-b824ee4b4f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:00.240494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30f72c-b3e6-4d67-964e-b824ee4b4f9b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:00.246886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 99/208] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:00 2022] GET /identity/v3/role_assignments?user.id=515c859dfab34f5982478169a915d7c2&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=135b77755ad64d97bfb82803f1c574f6 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:00.525508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967294a-c30c-479d-9f1a-a3a81be36133 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:00.526021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967294a-c30c-479d-9f1a-a3a81be36133 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:00.526021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967294a-c30c-479d-9f1a-a3a81be36133 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:00.527516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 110/209] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:00.532964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620ec70-a09e-4939-86c4-c31af94f0445 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:00.533256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620ec70-a09e-4939-86c4-c31af94f0445 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:00.533537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620ec70-a09e-4939-86c4-c31af94f0445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:00.558183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0620ec70-a09e-4939-86c4-c31af94f0445 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:00.559707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0620ec70-a09e-4939-86c4-c31af94f0445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:00.642131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 100/210] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:00.667192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967294a-c30c-479d-9f1a-a3a81be36133 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:00.706307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:00.780317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031fdd-3ef3-43ef-affd-abdd1293a7a6 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:00.780616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031fdd-3ef3-43ef-affd-abdd1293a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:00.780792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031fdd-3ef3-43ef-affd-abdd1293a7a6 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:00.782311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 101/211] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:00.787511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15916c25-40b3-451d-8d6f-9be0178ea96a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:00.788032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15916c25-40b3-451d-8d6f-9be0178ea96a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:00.788504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15916c25-40b3-451d-8d6f-9be0178ea96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:00.805326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967294a-c30c-479d-9f1a-a3a81be36133 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:00.810903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-15916c25-40b3-451d-8d6f-9be0178ea96a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:00.812089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-15916c25-40b3-451d-8d6f-9be0178ea96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:00.857410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:00.858314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:00.858821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:00.859337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:00.871719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:00.878497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:00.879555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:00.880336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:00.883341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 111/212] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:00.890252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:00.897012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 102/213] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:00.924892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15916c25-40b3-451d-8d6f-9be0178ea96a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:00.994899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:00.998019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:01.013700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565a54e-83c5-4b4d-8025-32b4a73d0c0d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.042479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15916c25-40b3-451d-8d6f-9be0178ea96a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.052781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.053699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.054157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.054722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.055710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:01.056188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:01.057900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:01.067089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 112/214] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:01.073401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:01.083175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.083829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.084107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.084377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.085668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:01.087798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:01.089578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 103/215] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:01.094618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:01.185947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdf215-945b-495a-9f04-6a3132cd0946 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.202956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a46fbe-bc8a-4959-8815-a9b65d0c2f70 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.224495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.225477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.225932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.226378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] PATH_INFO: `/v3/users/7cea77f702b5417f80a372f069133fc7` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.228976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:01.241599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf900b-3812-4bbe-a2b0-61c3ffe2ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.242246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf900b-3812-4bbe-a2b0-61c3ffe2ac4f None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.242498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf900b-3812-4bbe-a2b0-61c3ffe2ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.243364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf900b-3812-4bbe-a2b0-61c3ffe2ac4f None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.244051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf900b-3812-4bbe-a2b0-61c3ffe2ac4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:01.244753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] RBAC: Authorizing `identity:get_user(user_id=7cea77f702b5417f80a372f069133fc7)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:01.245545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf900b-3812-4bbe-a2b0-61c3ffe2ac4f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:01.246566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:01.248641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 113/216] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:01 2022] GET /identity/v3/users/7cea77f702b5417f80a372f069133fc7 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:01.266052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:01.282403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:01.290548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 104/217] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:01 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:17:01.338997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc0261-c7a7-4b20-b381-dd41f10c75ad None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.339641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc0261-c7a7-4b20-b381-dd41f10c75ad None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.340109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc0261-c7a7-4b20-b381-dd41f10c75ad None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.342079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 105/218] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:01.349425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c0e2c-53fd-4b79-91ec-0a00980bab72 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.350047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c0e2c-53fd-4b79-91ec-0a00980bab72 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.350513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c0e2c-53fd-4b79-91ec-0a00980bab72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.375531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d28c0e2c-53fd-4b79-91ec-0a00980bab72 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:01.377125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d28c0e2c-53fd-4b79-91ec-0a00980bab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:01.394214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c5a8-0d71-4029-b899-b347211e18ca None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.441964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9bfde-9898-4154-b228-bc1f9da692b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.443002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9bfde-9898-4154-b228-bc1f9da692b1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.443703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9bfde-9898-4154-b228-bc1f9da692b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.444622 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9bfde-9898-4154-b228-bc1f9da692b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.463968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 106/219] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:01.487954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c0e2c-53fd-4b79-91ec-0a00980bab72 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:01.533319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9bfde-9898-4154-b228-bc1f9da692b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:01.534803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9bfde-9898-4154-b228-bc1f9da692b1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:01.546070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:01.617574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c0e2c-53fd-4b79-91ec-0a00980bab72 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.631631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:01.662855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6c6ae-719a-45e4-a7a8-4e06a64c4dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.663686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6c6ae-719a-45e4-a7a8-4e06a64c4dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.663974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6c6ae-719a-45e4-a7a8-4e06a64c4dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.664250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6c6ae-719a-45e4-a7a8-4e06a64c4dc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.674917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 114/220] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 14:17:01.676536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb492-13eb-4b7e-9077-f334cc16f362 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.677123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb492-13eb-4b7e-9077-f334cc16f362 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.677412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb492-13eb-4b7e-9077-f334cc16f362 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.677595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6c6ae-719a-45e4-a7a8-4e06a64c4dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:01.678810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 115/221] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:01.683220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb492-13eb-4b7e-9077-f334cc16f362 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-91c6c6ae-719a-45e4-a7a8-4e06a64c4dc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:01.687256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:01.688965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:01.690545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 107/222] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:01.697121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570e126-3597-4cbc-a727-6c30bf7577e2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.697541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570e126-3597-4cbc-a727-6c30bf7577e2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.697947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570e126-3597-4cbc-a727-6c30bf7577e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.718958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8570e126-3597-4cbc-a727-6c30bf7577e2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:01.720211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8570e126-3597-4cbc-a727-6c30bf7577e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:01.796325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 108/223] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:01.797326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570e126-3597-4cbc-a727-6c30bf7577e2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:01.808809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb492-13eb-4b7e-9077-f334cc16f362 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.821679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:01.854629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.855319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.855598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.856493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.857741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:01.859731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:01.862412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 116/224] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:01 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:01.867229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:01.931676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:01.935360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570e126-3597-4cbc-a727-6c30bf7577e2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.982731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:01.983393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:01.983673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:01.983946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:01.984927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:01.985605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:01.987298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:01.988674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b2dd8-e0e8-4c9c-81d3-95f03ffbfd82 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:01.997677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 109/225] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:01.998582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:02.028154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.029112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.029593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.030057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.043435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:02.050751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:02.051858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:02.052858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 117/226] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:02.053629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:02.102381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:02.148729 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92709cd0-60ca-424c-abfa-13ce59bff08c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:02.196475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9a4d9-9a79-4fce-a3ca-7d50ca430d12 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:02.196890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.198044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.199454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.200005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.200992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=7cea77f702b5417f80a372f069133fc7) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:02.201558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:02.203223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:02.213200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 110/227] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:01 2022] GET /identity/v3/role_assignments?user.id=7cea77f702b5417f80a372f069133fc7&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:02.215195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:02.240245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.242000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.242659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.243170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] PATH_INFO: `/v3/users/82d41fd784e54d0ab1421c2da44dff16` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.245638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] RBAC: Authorizing `identity:get_user(user_id=82d41fd784e54d0ab1421c2da44dff16)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:02.247488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:02.251446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 118/228] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:02 2022] GET /identity/v3/users/82d41fd784e54d0ab1421c2da44dff16 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:02.260656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:02.267149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:02.423110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba5334-84c9-4fdc-bf9b-cf9af95311b7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:02.448778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fae13-b6a2-43cc-a62a-30ef4c31232e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:02.452397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:02.496120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.496743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.497018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.497288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.497993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:02.498293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:02.499676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:02.511624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 111/229] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:02.519370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.519516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:02.520027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.520309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.520585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.521769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:02.523259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:02.525498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 119/230] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:02 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:02.536683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:02.655600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:02.676885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c71d-ca75-465d-9d2b-26d34c0d2880 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:02.722924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.723503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.723785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.724060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] PATH_INFO: `/v3/users/82d41fd784e54d0ab1421c2da44dff16` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.726468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] RBAC: Authorizing `identity:get_user(user_id=82d41fd784e54d0ab1421c2da44dff16)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:02.728118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:02.729998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 112/231] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:02 2022] GET /identity/v3/users/82d41fd784e54d0ab1421c2da44dff16 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:02.739735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:02.753574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6db42-b208-4c44-bad0-f35a75b1014a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:02.821658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.822348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.822631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.822933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/82d41fd784e54d0ab1421c2da44dff16/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.830278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=923039d4dbc0458592a537e8bf17dd57, user_id=82d41fd784e54d0ab1421c2da44dff16, role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:02.832035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:02.870793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 120/232] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:02 2022] PUT /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/82d41fd784e54d0ab1421c2da44dff16/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:17:02.907543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e679de-df19-4149-922d-3f36c70af58f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:02.963126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee478d2-8413-4e0b-9c1f-85cbf314a7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:02.963794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee478d2-8413-4e0b-9c1f-85cbf314a7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:02.964072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee478d2-8413-4e0b-9c1f-85cbf314a7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:02.964346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee478d2-8413-4e0b-9c1f-85cbf314a7c4 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:02.965516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee478d2-8413-4e0b-9c1f-85cbf314a7c4 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:02.966985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee478d2-8413-4e0b-9c1f-85cbf314a7c4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:02.968641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 113/233] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:02 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:02.975278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:03.027284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:03.116050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e61a-d863-4c72-b999-3f796e6adbd8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:03.159814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d40c72-3c62-458f-9c6e-a7bf7d1b4911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:03.160509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40c72-3c62-458f-9c6e-a7bf7d1b4911 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:03.160794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40c72-3c62-458f-9c6e-a7bf7d1b4911 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:03.161073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40c72-3c62-458f-9c6e-a7bf7d1b4911 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/82d41fd784e54d0ab1421c2da44dff16/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:03.163545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d40c72-3c62-458f-9c6e-a7bf7d1b4911 None admin] RBAC: Authorizing `identity:create_grant(project_id=923039d4dbc0458592a537e8bf17dd57, user_id=82d41fd784e54d0ab1421c2da44dff16, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:03.165545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d40c72-3c62-458f-9c6e-a7bf7d1b4911 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:03.184526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 121/234] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:02 2022] PUT /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/82d41fd784e54d0ab1421c2da44dff16/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:03.751833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5ea0-2b8b-486d-a237-e4eaebe1d7d4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:03.752838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5ea0-2b8b-486d-a237-e4eaebe1d7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:03.753322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5ea0-2b8b-486d-a237-e4eaebe1d7d4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:03.755147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 114/235] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:03.760930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0483b-8d72-4fec-a7a3-369d79c9ad8a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:03.761753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0483b-8d72-4fec-a7a3-369d79c9ad8a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:03.762241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0483b-8d72-4fec-a7a3-369d79c9ad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:03.783715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-84e0483b-8d72-4fec-a7a3-369d79c9ad8a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:03.785218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-84e0483b-8d72-4fec-a7a3-369d79c9ad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:03.841130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96d44-70ad-48b6-ad45-dea1153b943d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:03.841426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96d44-70ad-48b6-ad45-dea1153b943d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:03.841678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96d44-70ad-48b6-ad45-dea1153b943d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:03.843432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 115/236] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:03.851877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4bc04-3332-4fb1-95d4-306a44b263ee None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:03.852786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4bc04-3332-4fb1-95d4-306a44b263ee None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:03.853084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4bc04-3332-4fb1-95d4-306a44b263ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:03.870831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1bb4bc04-3332-4fb1-95d4-306a44b263ee None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:03.872050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1bb4bc04-3332-4fb1-95d4-306a44b263ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:03.889539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:03.907100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 122/237] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:03.939290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 116/238] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:03.954325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0483b-8d72-4fec-a7a3-369d79c9ad8a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:03.962226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4bc04-3332-4fb1-95d4-306a44b263ee None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:04.002603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:04.062055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4bc04-3332-4fb1-95d4-306a44b263ee None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:04.095414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:04.096064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.096348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.096625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.107466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:04.112319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:04.113185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:04.113975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 117/239] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:04.119728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:04.151194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:04.220974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151cc8-2658-444e-9d45-1a08d1b53a1a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:04.258809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:04.259381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.259489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0483b-8d72-4fec-a7a3-369d79c9ad8a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:04.259650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.259930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.260653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:04.261116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:04.262531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:04.272046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 118/240] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:04.278466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:04.359803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:04.360454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.360770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.361044 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.362194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:04.363657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:04.365247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 123/241] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:04.370637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:04.375214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:04.429014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4786ae7-27c5-46a8-9fc5-ca2406e07f69 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:04.490493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:04.501744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22a9da-431c-4ed3-8826-4459346e39f4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:04.513587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:04.514182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.514464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.514736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] PATH_INFO: `/v3/users/f44f6b36ea1040ae91bf6349dafabac2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.543246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] RBAC: Authorizing `identity:get_user(user_id=f44f6b36ea1040ae91bf6349dafabac2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:04.545521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:04.550125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 119/242] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:04 2022] GET /identity/v3/users/f44f6b36ea1040ae91bf6349dafabac2 => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:04.555625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:04.562717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eb9f8-5a1f-464a-bf4f-86033587929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:04.563372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eb9f8-5a1f-464a-bf4f-86033587929d None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.563646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eb9f8-5a1f-464a-bf4f-86033587929d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.563912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eb9f8-5a1f-464a-bf4f-86033587929d None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.564876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206eb9f8-5a1f-464a-bf4f-86033587929d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:04.566390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206eb9f8-5a1f-464a-bf4f-86033587929d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:04.609730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 124/243] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:04 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:17:04.700373 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26a79-6f21-48c5-917e-8c1c81215bd2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.701475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26a79-6f21-48c5-917e-8c1c81215bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.701475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26a79-6f21-48c5-917e-8c1c81215bd2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.702819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 125/244] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:04.712049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0e1a-0b8c-4efd-a22f-7c672d0b3762 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:04.720636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3031fc5-7d66-4c2f-8de6-574fd857d192 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.720939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3031fc5-7d66-4c2f-8de6-574fd857d192 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.721198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3031fc5-7d66-4c2f-8de6-574fd857d192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.762015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b3031fc5-7d66-4c2f-8de6-574fd857d192 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:04.764959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b3031fc5-7d66-4c2f-8de6-574fd857d192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:04.767495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:04.768711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:04.769226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:04.769695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:04.775413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:04.887915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:04.889871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:04.898564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 126/245] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:04.930080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3031fc5-7d66-4c2f-8de6-574fd857d192 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.021010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:05.026959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 120/246] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:05.033413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.053561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3031fc5-7d66-4c2f-8de6-574fd857d192 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:05.069878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:05.098576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:05.098779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.099378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.099986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.114487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:05.119669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:05.120525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:05.121687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 127/247] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:05.126958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.149410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ce19d-b7e7-4480-95e3-2c6390e796a2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:05.189773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:05.190750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.191382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.191927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.194392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:05.196192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:05.198710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 121/248] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:05 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:05.204545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.242627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362278-75f3-4f2b-bac0-d32d9add8211 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:05.283886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:05.284484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.284771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.285041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.285869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:05.286195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:05.287715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:05.297503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 128/249] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:05.298084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:05.303105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.341635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e5e1d-a221-4290-96de-cb5cf35a0a6c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:05.344173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:05.382286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:05.382971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.383349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.383628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.384460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] RBAC: Adding query filter params (role.id=135b77755ad64d97bfb82803f1c574f6, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=f44f6b36ea1040ae91bf6349dafabac2) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:05.384770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:05.386255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:05.392101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 122/250] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:05 2022] GET /identity/v3/role_assignments?user.id=f44f6b36ea1040ae91bf6349dafabac2&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=135b77755ad64d97bfb82803f1c574f6 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:05.437814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fb571-19db-4aad-b3ec-a5f0ad4bdbb5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:05.480003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf865c98-4c26-4fe3-8d95-34f86f3d88ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:05.481094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf865c98-4c26-4fe3-8d95-34f86f3d88ab None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.481588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf865c98-4c26-4fe3-8d95-34f86f3d88ab None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.482125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf865c98-4c26-4fe3-8d95-34f86f3d88ab None admin] PATH_INFO: `/v3/users/7cea77f702b5417f80a372f069133fc7` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.483871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf865c98-4c26-4fe3-8d95-34f86f3d88ab None admin] RBAC: Authorizing `identity:get_user(user_id=7cea77f702b5417f80a372f069133fc7)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:05.485713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf865c98-4c26-4fe3-8d95-34f86f3d88ab None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:05.487854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 129/251] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:05 2022] GET /identity/v3/users/7cea77f702b5417f80a372f069133fc7 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:05.497995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.515947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739f577-ab38-4b66-8824-2d33925398a6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.516253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739f577-ab38-4b66-8824-2d33925398a6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.516507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739f577-ab38-4b66-8824-2d33925398a6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.518037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 130/252] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:05.524548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a42c2b-b750-4ef6-82bd-6b84b8dbd941 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.524836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a42c2b-b750-4ef6-82bd-6b84b8dbd941 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.525113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a42c2b-b750-4ef6-82bd-6b84b8dbd941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.558709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d5a42c2b-b750-4ef6-82bd-6b84b8dbd941 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:05.560033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d5a42c2b-b750-4ef6-82bd-6b84b8dbd941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:05.669281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:05.689584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 131/253] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:05.708432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb2fd2-d183-44a0-9b3a-a83f14528678 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:05.739995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a42c2b-b750-4ef6-82bd-6b84b8dbd941 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.779391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:05.780054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.780335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.780609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.782499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:05.783972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:05.785966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 123/254] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:05 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:05.791912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:05.859597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:05.893128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a42c2b-b750-4ef6-82bd-6b84b8dbd941 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:05.932812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d708e2-dc2f-4207-a2b7-ca63713d943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:05.933474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d708e2-dc2f-4207-a2b7-ca63713d943f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.933751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d708e2-dc2f-4207-a2b7-ca63713d943f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.934023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d708e2-dc2f-4207-a2b7-ca63713d943f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.952738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d708e2-dc2f-4207-a2b7-ca63713d943f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-32d708e2-dc2f-4207-a2b7-ca63713d943f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:05.959220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:05.960316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:05.961356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:05.964328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 132/255] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:05.964608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a37a-831f-4009-947f-bb8547811708 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.965077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a37a-831f-4009-947f-bb8547811708 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.965427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a37a-831f-4009-947f-bb8547811708 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.968523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 133/256] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:05.972513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e037704-d1f9-4514-baba-4022ac289deb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:05.972864 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e037704-d1f9-4514-baba-4022ac289deb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:05.973214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e037704-d1f9-4514-baba-4022ac289deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:05.993822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5e037704-d1f9-4514-baba-4022ac289deb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:05.995175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5e037704-d1f9-4514-baba-4022ac289deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:06.004992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4d2a9-6386-495a-a727-3400ef659d0a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.048317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:06.069532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 134/257] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:06.070404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e037704-d1f9-4514-baba-4022ac289deb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:06.093997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.094684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.094986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.095263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/7cea77f702b5417f80a372f069133fc7/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:06.098399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] RBAC: Authorizing `identity:create_grant(project_id=923039d4dbc0458592a537e8bf17dd57, user_id=7cea77f702b5417f80a372f069133fc7, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:06.099879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:06.127407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 124/258] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:05 2022] PUT /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/7cea77f702b5417f80a372f069133fc7/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:17:06.128220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:06.233232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e037704-d1f9-4514-baba-4022ac289deb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.309156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.310058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.310349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.310626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:06.311389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:06.311763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:06.314166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:06.333711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 135/259] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:06.340749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:06.366711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:06.383656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f87e9b-b70c-43e9-b8f8-fbf0d7ae7443 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.433715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.434368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.434645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.434954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:06.437006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:06.446703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:06.452134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:06.453010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:06.453789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 125/260] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:06.459483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:06.553020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7583bc-af17-4450-a587-0287ad9402c3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.563871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bb666-3786-44df-8359-f541b751b4d9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.599104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.599730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.600005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.600274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:06.600986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:06.601285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:06.603315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:06.611786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 126/261] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:06.617140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:06.635491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:06.644901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.645630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.645914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.646189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] PATH_INFO: `/v3/users/82d41fd784e54d0ab1421c2da44dff16` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:06.651588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] RBAC: Authorizing `identity:get_user(user_id=82d41fd784e54d0ab1421c2da44dff16)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:06.653059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:06.655037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 136/262] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:06 2022] GET /identity/v3/users/82d41fd784e54d0ab1421c2da44dff16 => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:06.660670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:06.722748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34d635-a6c1-4c41-811d-83cf5450bfb9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.759233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.760186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.760470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.760745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] PATH_INFO: `/v3/users/82d41fd784e54d0ab1421c2da44dff16` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:06.762063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] RBAC: Authorizing `identity:get_user(user_id=82d41fd784e54d0ab1421c2da44dff16)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:06.763549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:06.765303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 127/263] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:06 2022] GET /identity/v3/users/82d41fd784e54d0ab1421c2da44dff16 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:06.772891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:06.878123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:06.894509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a800f07-0c9e-4408-999f-6322df9185ff None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.926248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455fc12-b2f6-41af-bfcc-4f879195f022 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:06.930538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.931211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.931491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.931762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:06.960475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:06.986248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:06.987209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:06.987489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:06.987766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:07.028544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:07.030026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:07.116427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:07.118140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:07.175328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:07.210645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 128/264] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:07.219810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:07.269463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 137/265] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 14:17:07.276585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:07.303190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:07.450377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef190b5-1ec2-4fe7-857b-1d9edb93060f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:07.458625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b867fc9-9e03-42fd-ab4c-5c1359c080ed None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:07.486706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:07.487388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:07.487759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:07.488035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:07.489295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:07.490738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:07.492954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 138/266] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:07 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:07.499893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:07.514383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:07.515075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:07.515359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:07.515634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:07.516808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:07.518254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:07.520269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 129/267] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:07 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:07.541205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:07.595814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:07.612563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1818e8-0997-46ab-9bb5-118bea99b190 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:07.631929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:07.671744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11884665-dbfc-4f39-875b-7d67f022a573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:07.672373 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11884665-dbfc-4f39-875b-7d67f022a573 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:07.672648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11884665-dbfc-4f39-875b-7d67f022a573 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:07.672915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11884665-dbfc-4f39-875b-7d67f022a573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:07.673702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11884665-dbfc-4f39-875b-7d67f022a573 None admin] RBAC: Adding query filter params (role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=82d41fd784e54d0ab1421c2da44dff16) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:07.674007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11884665-dbfc-4f39-875b-7d67f022a573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:07.677621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11884665-dbfc-4f39-875b-7d67f022a573 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:07.688201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 139/268] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:07 2022] GET /identity/v3/role_assignments?user.id=82d41fd784e54d0ab1421c2da44dff16&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:07.707215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515033d-33b9-4bb2-b020-bcb85ccaf1dc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:07.745013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dd1dd-128a-4809-9deb-f2b8633ea1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:07.745645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dd1dd-128a-4809-9deb-f2b8633ea1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:07.745920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dd1dd-128a-4809-9deb-f2b8633ea1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:07.746188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dd1dd-128a-4809-9deb-f2b8633ea1b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:07.747075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dd1dd-128a-4809-9deb-f2b8633ea1b6 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=82d41fd784e54d0ab1421c2da44dff16) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:07.747385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dd1dd-128a-4809-9deb-f2b8633ea1b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:07.748983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dd1dd-128a-4809-9deb-f2b8633ea1b6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:07.759712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 130/269] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:07 2022] GET /identity/v3/role_assignments?user.id=82d41fd784e54d0ab1421c2da44dff16&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:08.146277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92cba-78dd-49d6-ae25-ce5a1c93262a None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.146789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92cba-78dd-49d6-ae25-ce5a1c93262a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.146904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92cba-78dd-49d6-ae25-ce5a1c93262a None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.148516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 140/270] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:08.154191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862fcfe-3eda-490a-9eca-183c9202d213 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.154601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862fcfe-3eda-490a-9eca-183c9202d213 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.154788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862fcfe-3eda-490a-9eca-183c9202d213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.175682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4862fcfe-3eda-490a-9eca-183c9202d213 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:08.177209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4862fcfe-3eda-490a-9eca-183c9202d213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:08.259973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 131/271] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:08.285663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e92cba-78dd-49d6-ae25-ce5a1c93262a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:08.303723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37266b4e-a28d-4173-9744-7867265db7fd None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.304340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37266b4e-a28d-4173-9744-7867265db7fd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.304981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37266b4e-a28d-4173-9744-7867265db7fd None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.306608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 132/272] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:08.312080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1dd7d-dccd-4e8f-bc19-c7bb3796a42f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.312367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1dd7d-dccd-4e8f-bc19-c7bb3796a42f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.312644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1dd7d-dccd-4e8f-bc19-c7bb3796a42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.340447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-33e1dd7d-dccd-4e8f-bc19-c7bb3796a42f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:08.341658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-33e1dd7d-dccd-4e8f-bc19-c7bb3796a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:08.393606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:08.417926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e92cba-78dd-49d6-ae25-ce5a1c93262a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:08.420543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 133/273] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:08.421774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3b4d2-846f-436d-ab7a-9fa73db8896e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.422061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3b4d2-846f-436d-ab7a-9fa73db8896e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.422336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3b4d2-846f-436d-ab7a-9fa73db8896e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.423740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 134/274] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:08.429075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6d98b-cfaa-4d5d-a3f6-e411303c1f2c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.429362 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6d98b-cfaa-4d5d-a3f6-e411303c1f2c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.429634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6d98b-cfaa-4d5d-a3f6-e411303c1f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.455449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1af6d98b-cfaa-4d5d-a3f6-e411303c1f2c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:08.456415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:08.456644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1af6d98b-cfaa-4d5d-a3f6-e411303c1f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:08.457054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.457330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.457602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.468642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:08.473556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:08.474403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:08.475223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 141/275] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:08.475860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:08.486310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:08.529208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 135/276] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:08.536037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6d98b-cfaa-4d5d-a3f6-e411303c1f2c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:08.587743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd6cd9-ee00-4ddf-bfdb-ecd896be0f0c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:08.622708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:08.623378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.623659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.623936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.625084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:08.626533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:08.628163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 142/277] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:08.628923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:08.670227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6d98b-cfaa-4d5d-a3f6-e411303c1f2c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:08.678736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:08.706895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:08.707542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.707823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.708097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.708806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:08.709108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:08.709884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:08.710463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:08.719114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 136/278] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:08.719981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:08.735613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfaa55-e3dc-4693-999a-94d15e6b5067 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:08.771396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:08.772070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.772354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.772629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.784384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:08.792185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:08.793063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:08.793833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 143/279] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:08.795077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:08.830810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5f46-41b9-4543-a2a3-226e0fc03bbe None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:08.873195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:08.873827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.874499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.874784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.875821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:08.877519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:08.897552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770465f-26fd-4050-8a67-c0efaf84f8de None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:08.902899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 137/280] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:08 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:08.912317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:08.938740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:08.939402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:08.939673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:08.940056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] PATH_INFO: `/v3/users/2f8e2d51704d418085a6b37077f56a7e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:08.944737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:08.955239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:08.963210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] RBAC: Authorizing `identity:get_user(user_id=2f8e2d51704d418085a6b37077f56a7e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:08.966000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:08.968113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 144/281] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:08 2022] GET /identity/v3/users/2f8e2d51704d418085a6b37077f56a7e => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:08.975481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:09.049040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4735481-46d3-443d-ad88-63e4c21fdd0f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.088595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6c8bd-e62d-48a4-8708-15a46dfe6c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.089229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6c8bd-e62d-48a4-8708-15a46dfe6c3c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.089527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6c8bd-e62d-48a4-8708-15a46dfe6c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.089787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6c8bd-e62d-48a4-8708-15a46dfe6c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.090252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6889371-2fb6-4af2-b95a-bbddb53be200 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.090506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6c8bd-e62d-48a4-8708-15a46dfe6c3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:09.090813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6c8bd-e62d-48a4-8708-15a46dfe6c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:09.093375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6c8bd-e62d-48a4-8708-15a46dfe6c3c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:09.101780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 138/282] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:09.103577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859ecf4-f2fe-43c2-beac-f730f39ac596 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.103864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859ecf4-f2fe-43c2-beac-f730f39ac596 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.104145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859ecf4-f2fe-43c2-beac-f730f39ac596 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.105366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 139/283] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:09.110028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859ecf4-f2fe-43c2-beac-f730f39ac596 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:09.130557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2eae4-7138-482d-8eb3-1a3242e92dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.131215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2eae4-7138-482d-8eb3-1a3242e92dea None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.131490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2eae4-7138-482d-8eb3-1a3242e92dea None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.131752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2eae4-7138-482d-8eb3-1a3242e92dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.132539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2eae4-7138-482d-8eb3-1a3242e92dea None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, user.id=2f8e2d51704d418085a6b37077f56a7e, scope.system=all) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:09.132853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2eae4-7138-482d-8eb3-1a3242e92dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:09.134320 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2eae4-7138-482d-8eb3-1a3242e92dea None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:09.139532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 145/284] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Aug 6 14:17:08 2022] GET /identity/v3/role_assignments?user.id=2f8e2d51704d418085a6b37077f56a7e&scope.system=all&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:09.140618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb60e4-5ff1-4842-abe4-9bee680b68cd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.140909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb60e4-5ff1-4842-abe4-9bee680b68cd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.141190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb60e4-5ff1-4842-abe4-9bee680b68cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.163361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-75bb60e4-5ff1-4842-abe4-9bee680b68cd None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:09.165156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-75bb60e4-5ff1-4842-abe4-9bee680b68cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:09.230826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:09.236195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:09.255509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 146/285] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:09.256486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859ecf4-f2fe-43c2-beac-f730f39ac596 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.291187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb60e4-5ff1-4842-abe4-9bee680b68cd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:09.303768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.304441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.304725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.305514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] PATH_INFO: `/v3/users/f44f6b36ea1040ae91bf6349dafabac2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.308384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] RBAC: Authorizing `identity:get_user(user_id=f44f6b36ea1040ae91bf6349dafabac2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:09.308966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:09.311782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 140/286] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:09 2022] GET /identity/v3/users/f44f6b36ea1040ae91bf6349dafabac2 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:09.317634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:09.418777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb60e4-5ff1-4842-abe4-9bee680b68cd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.444682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9cbbe-9896-4952-bc57-59ebb68fcb88 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.463231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.464383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.464717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.465135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.488255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:09.495190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.495834 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.496117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.496392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.497573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:09.499043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:09.502123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:09.503011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:09.503792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 147/287] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:09.504427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 141/288] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:09 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:09.511111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:09.514342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:09.563664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:09.625425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7664c4-833b-4eb5-b83e-4c4f22817a89 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.652795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:09.668045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.668731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.669010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.669284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/f44f6b36ea1040ae91bf6349dafabac2/roles/135b77755ad64d97bfb82803f1c574f6` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.671948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] RBAC: Authorizing `identity:create_grant(project_id=923039d4dbc0458592a537e8bf17dd57, user_id=f44f6b36ea1040ae91bf6349dafabac2, role_id=135b77755ad64d97bfb82803f1c574f6)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:09.673390 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:09.694281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 142/289] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:09 2022] PUT /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57/users/f44f6b36ea1040ae91bf6349dafabac2/roles/135b77755ad64d97bfb82803f1c574f6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:17:09.733305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71877248-8e93-49ba-9812-2a70f200d032 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.772402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.773001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.773282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.773641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.774378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:09.774680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:09.776131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:09.794609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 148/290] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:09.800901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:09.917985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:09.941919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227ff56-62cd-4ce2-9840-bf489e1adedc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:09.979187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:09.980229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:09.980728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:09.981187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] PATH_INFO: `/v3/users/7cea77f702b5417f80a372f069133fc7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:09.982556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=7cea77f702b5417f80a372f069133fc7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:09.984228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:09.986111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 143/291] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:09 2022] GET /identity/v3/users/7cea77f702b5417f80a372f069133fc7 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:09.992022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:10.131920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9586faa-6e69-4805-a545-41f9db68aace None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:10.146059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:10.182499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:10.183155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:10.183412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:10.183685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:10.278296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:10.279796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:10.399444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 149/292] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:10.405238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:10.538495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31769361-279c-4e1e-a87c-cd5351e97bf7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:10.577052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c17cc-fa52-4e9f-aab2-28e5da9a64f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:10.577698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c17cc-fa52-4e9f-aab2-28e5da9a64f1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:10.577979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c17cc-fa52-4e9f-aab2-28e5da9a64f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:10.578312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c17cc-fa52-4e9f-aab2-28e5da9a64f1 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:10.579504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c17cc-fa52-4e9f-aab2-28e5da9a64f1 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:10.580939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c17cc-fa52-4e9f-aab2-28e5da9a64f1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:10.582599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 144/293] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:10 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:10.588734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:10.603890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:10.700515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6720f2-82ad-4ef4-b63f-751d9e3fa694 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:10.736236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f3010-3a0e-4119-89e3-c2c19e418d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:10.736910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f3010-3a0e-4119-89e3-c2c19e418d60 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:10.737192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f3010-3a0e-4119-89e3-c2c19e418d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:10.737472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f3010-3a0e-4119-89e3-c2c19e418d60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:10.738316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f3010-3a0e-4119-89e3-c2c19e418d60 None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=7cea77f702b5417f80a372f069133fc7) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:10.738625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f3010-3a0e-4119-89e3-c2c19e418d60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:10.740180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f3010-3a0e-4119-89e3-c2c19e418d60 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:10.746452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 150/294] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:10 2022] GET /identity/v3/role_assignments?user.id=7cea77f702b5417f80a372f069133fc7&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:11.527565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e41023-af7f-4d54-8da6-b6c98e1245f1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.528087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e41023-af7f-4d54-8da6-b6c98e1245f1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.528087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e41023-af7f-4d54-8da6-b6c98e1245f1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.529692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 145/295] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:11.534931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f6225-2e50-47ec-81c1-90262e929312 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.535755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f6225-2e50-47ec-81c1-90262e929312 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.536223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f6225-2e50-47ec-81c1-90262e929312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.554653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-876f6225-2e50-47ec-81c1-90262e929312 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:11.556175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-876f6225-2e50-47ec-81c1-90262e929312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:11.626497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 151/296] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:11.649373 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e41023-af7f-4d54-8da6-b6c98e1245f1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:11.669701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3696698-3a00-474c-9f47-d20e433588ba None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.670301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3696698-3a00-474c-9f47-d20e433588ba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.670770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3696698-3a00-474c-9f47-d20e433588ba None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.672379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:11.672703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 152/297] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:11.680073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d7f4e-7749-4c93-8133-83c785b3bd6f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.680638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d7f4e-7749-4c93-8133-83c785b3bd6f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.681124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d7f4e-7749-4c93-8133-83c785b3bd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.689196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:11.703680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8a6d7f4e-7749-4c93-8133-83c785b3bd6f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:11.705205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8a6d7f4e-7749-4c93-8133-83c785b3bd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:11.760171 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e41023-af7f-4d54-8da6-b6c98e1245f1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:11.790924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 153/298] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:11.790924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c5dd-c6cf-4705-b6c7-6d2285453938 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.790924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c5dd-c6cf-4705-b6c7-6d2285453938 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.790924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c5dd-c6cf-4705-b6c7-6d2285453938 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.790924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 154/299] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:11.790924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa87cad-f579-4f36-9672-3353771a55fb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.790924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa87cad-f579-4f36-9672-3353771a55fb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.791412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa87cad-f579-4f36-9672-3353771a55fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.803727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:11.804371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.804848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.805109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.808471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2fa87cad-f579-4f36-9672-3353771a55fb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:11.809647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2fa87cad-f579-4f36-9672-3353771a55fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:11.815839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:11.821166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:11.822010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:11.822783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 146/300] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:11.823489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:11.888147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 155/301] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:11.895736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa87cad-f579-4f36-9672-3353771a55fb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:11.916821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:11.940567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6de3d-8c6f-4afb-a8a3-64f0c7765337 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:11.961923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:11.975399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:11.976044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:11.976321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:11.976591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:11.988268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:11.993659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:11.994513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 147/302] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:11.995773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.003954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa87cad-f579-4f36-9672-3353771a55fb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.037577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65eef6-ea09-49f4-bd24-13852d130951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.038216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65eef6-ea09-49f4-bd24-13852d130951 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.038490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65eef6-ea09-49f4-bd24-13852d130951 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.038763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65eef6-ea09-49f4-bd24-13852d130951 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.040062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65eef6-ea09-49f4-bd24-13852d130951 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:12.040365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65eef6-ea09-49f4-bd24-13852d130951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.041769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65eef6-ea09-49f4-bd24-13852d130951 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.049654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 156/303] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:12.050789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731a21b-2c94-4988-973f-3a5ec154fd98 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.051100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731a21b-2c94-4988-973f-3a5ec154fd98 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.051470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731a21b-2c94-4988-973f-3a5ec154fd98 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.052627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 157/304] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:12.053231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731a21b-2c94-4988-973f-3a5ec154fd98 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.092351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e4ae6-f3d5-4731-9c05-2009a4a44fea None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.125710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.126342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.126619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.126909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.128026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.129501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.131095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 148/305] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:12.131841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.140430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:12.154951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731a21b-2c94-4988-973f-3a5ec154fd98 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.180137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:12.188833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5713da-7fa7-4ed5-8d6f-f05a297435cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.189455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5713da-7fa7-4ed5-8d6f-f05a297435cd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.189738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5713da-7fa7-4ed5-8d6f-f05a297435cd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.190011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5713da-7fa7-4ed5-8d6f-f05a297435cd None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.190756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5713da-7fa7-4ed5-8d6f-f05a297435cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:12.191102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5713da-7fa7-4ed5-8d6f-f05a297435cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.192473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5713da-7fa7-4ed5-8d6f-f05a297435cd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.201085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 158/306] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:12.202208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fd5e9-f6ca-4350-87b1-6d44b19605b1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.202498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fd5e9-f6ca-4350-87b1-6d44b19605b1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.202770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fd5e9-f6ca-4350-87b1-6d44b19605b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.220898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4a3fd5e9-f6ca-4350-87b1-6d44b19605b1 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:12.222076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4a3fd5e9-f6ca-4350-87b1-6d44b19605b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:12.236981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b41876-2262-4f68-867a-f62a5889c48e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.271609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.272244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.272523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.272802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] PATH_INFO: `/v3/users/2f8e2d51704d418085a6b37077f56a7e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.274555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] RBAC: Authorizing `identity:get_user(user_id=2f8e2d51704d418085a6b37077f56a7e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.276029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.277793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 149/307] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:12 2022] GET /identity/v3/users/2f8e2d51704d418085a6b37077f56a7e => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:12.278334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.290015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 159/308] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:12.290856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fd5e9-f6ca-4350-87b1-6d44b19605b1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.360432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:12.385854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895c25f-2f5a-489b-a718-bcf7cd0d3c7a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.386317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fd5e9-f6ca-4350-87b1-6d44b19605b1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.404214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:12.419208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.419820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.420097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.420366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] PATH_INFO: `/v3/groups` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.420823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.421009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.421457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.421737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.422012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] PATH_INFO: `/v3/users/307ce9a52e08458b8d5126d98e1b9b57` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.422414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.435836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] RBAC: Authorizing `identity:get_user(user_id=307ce9a52e08458b8d5126d98e1b9b57)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.437293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.437942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 150/309] 10.222.0.17 () {60 vars in 1134 bytes} [Sat Aug 6 14:17:12 2022] POST /identity/v3/groups => generated 229 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:12.444140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 160/310] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:12 2022] GET /identity/v3/users/307ce9a52e08458b8d5126d98e1b9b57 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:12.444325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.444414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.558481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375e4f-ddd3-4c21-9550-9efffbac96c5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.565263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8cb0-636e-4484-a9fa-99b857822f74 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.596446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.596648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.596911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.597182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.604071 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.604739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.605019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.605295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] PATH_INFO: `/v3/system/users/2f8e2d51704d418085a6b37077f56a7e/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.606752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f8e2d51704d418085a6b37077f56a7e, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.608163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.609712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:12.611833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:12.618613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 151/311] 10.222.0.17 () {58 vars in 1334 bytes} [Sat Aug 6 14:17:12 2022] PUT /identity/v3/system/users/2f8e2d51704d418085a6b37077f56a7e/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:12.618764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:12.619680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:12.620441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 161/312] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:12.624074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.627275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.691247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:12.763566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb1221-c974-42a0-bfe0-dbf63f1dd337 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.787736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749aab-0021-45b5-9631-656064b6b69c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:12.804265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.805438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.805720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.805994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.807197 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] RBAC: Adding query filter params (role.id=135b77755ad64d97bfb82803f1c574f6, user.id=307ce9a52e08458b8d5126d98e1b9b57, scope.system=all) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:12.807514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.809234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.815712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 152/313] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Aug 6 14:17:12 2022] GET /identity/v3/role_assignments?user.id=307ce9a52e08458b8d5126d98e1b9b57&scope.system=all&role.id=135b77755ad64d97bfb82803f1c574f6 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:12.828621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:12.829254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:12.829531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:12.829801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:12.830515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:12.830811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:12.832326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:12.852305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 162/314] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:12.857881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:12.934101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:12.978567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ed20-5ba6-428e-9df6-e7901fecbc84 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:13.015485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:13.016136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:13.016415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:13.016693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] PATH_INFO: `/v3/users/f44f6b36ea1040ae91bf6349dafabac2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:13.017973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] RBAC: Authorizing `identity:get_user(user_id=f44f6b36ea1040ae91bf6349dafabac2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:13.019501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:13.021464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 153/315] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:12 2022] GET /identity/v3/users/f44f6b36ea1040ae91bf6349dafabac2 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:13.028828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:13.143330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:13.148230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756015-68ca-44c2-877a-9ff023135926 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:13.185681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:13.186226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:13.186504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:13.186774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:13.257071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:13.258566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:13.357826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:13.365356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 163/316] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:13.371753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:13.483976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775c6e-2dd2-4590-adae-ad1629f5ac5e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:13.513743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:13.521029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4575d-c1bd-4a5a-84ca-80723bbd6ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:13.521686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4575d-c1bd-4a5a-84ca-80723bbd6ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:13.521967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4575d-c1bd-4a5a-84ca-80723bbd6ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:13.522251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4575d-c1bd-4a5a-84ca-80723bbd6ff9 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:13.523433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4575d-c1bd-4a5a-84ca-80723bbd6ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:13.524884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4575d-c1bd-4a5a-84ca-80723bbd6ff9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:13.526826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 154/317] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:13 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:13.532670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:13.637722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471147c-9635-4f44-8787-49c869d85cd5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:13.678006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a7e43-7815-4b98-a975-ea7055ffb64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:13.678634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a7e43-7815-4b98-a975-ea7055ffb64f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:13.679232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a7e43-7815-4b98-a975-ea7055ffb64f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:13.679504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a7e43-7815-4b98-a975-ea7055ffb64f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:13.680947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a7e43-7815-4b98-a975-ea7055ffb64f None admin] RBAC: Adding query filter params (role.id=135b77755ad64d97bfb82803f1c574f6, scope.project.id=923039d4dbc0458592a537e8bf17dd57, user.id=f44f6b36ea1040ae91bf6349dafabac2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:13.681255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a7e43-7815-4b98-a975-ea7055ffb64f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:13.683344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a7e43-7815-4b98-a975-ea7055ffb64f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:13.692925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 164/318] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:13 2022] GET /identity/v3/role_assignments?user.id=f44f6b36ea1040ae91bf6349dafabac2&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=135b77755ad64d97bfb82803f1c574f6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:14.530698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f93cd7-0dac-4ef8-943f-094ed148051a None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.531238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f93cd7-0dac-4ef8-943f-094ed148051a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.531347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f93cd7-0dac-4ef8-943f-094ed148051a None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.532635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 155/319] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:14.537708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2a639-f610-45f6-ab48-6487143bb4e0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.537998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2a639-f610-45f6-ab48-6487143bb4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.538275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2a639-f610-45f6-ab48-6487143bb4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.557513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cba2a639-f610-45f6-ab48-6487143bb4e0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:14.558681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cba2a639-f610-45f6-ab48-6487143bb4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:14.615403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:14.626547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 165/320] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:14.647830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f93cd7-0dac-4ef8-943f-094ed148051a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:14.702070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455af1e-d77a-4863-8838-8cbebf898da1 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.702569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455af1e-d77a-4863-8838-8cbebf898da1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.702641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455af1e-d77a-4863-8838-8cbebf898da1 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.704103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 166/321] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:14.708749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c56ff3-0ebd-4719-8b99-7db6b9269562 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.709034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c56ff3-0ebd-4719-8b99-7db6b9269562 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.709309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c56ff3-0ebd-4719-8b99-7db6b9269562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.727931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d8c56ff3-0ebd-4719-8b99-7db6b9269562 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:14.729887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d8c56ff3-0ebd-4719-8b99-7db6b9269562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:14.754970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f93cd7-0dac-4ef8-943f-094ed148051a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:14.789756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:14.790337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.790617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.790905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.792060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:14.793492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:14.795132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 156/322] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:14.801390 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:14.807899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 167/323] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:14.829971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c56ff3-0ebd-4719-8b99-7db6b9269562 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:14.873545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:14.907896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1121f-cd9b-4ee3-94fe-ea487acb4c29 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:14.941627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e8f9f-6e3c-4e2b-8e38-31d6bd0506db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:14.942245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8f9f-6e3c-4e2b-8e38-31d6bd0506db None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.942529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8f9f-6e3c-4e2b-8e38-31d6bd0506db None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.942802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8f9f-6e3c-4e2b-8e38-31d6bd0506db None admin] PATH_INFO: `/v3/groups` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.943262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c56ff3-0ebd-4719-8b99-7db6b9269562 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:14.943486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e8f9f-6e3c-4e2b-8e38-31d6bd0506db None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:14.944904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e8f9f-6e3c-4e2b-8e38-31d6bd0506db None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:14.959301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 157/324] 10.222.0.17 () {60 vars in 1134 bytes} [Sat Aug 6 14:17:14 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:17:14.960562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03b8bb-b540-4045-ab7b-63ea2695a3b1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.960868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03b8bb-b540-4045-ab7b-63ea2695a3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.961657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03b8bb-b540-4045-ab7b-63ea2695a3b1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.963044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 158/325] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:14.968676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:14.968861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462745e0-7adc-43a9-a590-61fa63c44f90 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.969091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462745e0-7adc-43a9-a590-61fa63c44f90 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.969316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462745e0-7adc-43a9-a590-61fa63c44f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.983093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:14.983767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:14.984043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:14.984323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:14.989746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-462745e0-7adc-43a9-a590-61fa63c44f90 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:14.991349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-462745e0-7adc-43a9-a590-61fa63c44f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:14.996147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:15.001211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:15.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:15.003175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 168/326] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:15.008446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:15.069650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 159/327] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:15.093709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462745e0-7adc-43a9-a590-61fa63c44f90 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:15.131694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581708ca-1485-4bad-a691-27168b2ce0af None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:15.155730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:15.173060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:15.173699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:15.173979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:15.174257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:15.175272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:15.175571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:15.176952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:15.185826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 169/328] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:15.192290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:15.218319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462745e0-7adc-43a9-a590-61fa63c44f90 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:15.255354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:15.265681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:15.266787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:15.267119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:15.267667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:15.283394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:15.290464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:15.291382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:15.292155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:15.297816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 160/329] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:15.299731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:15.322130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7f282-e549-48af-bf27-51ac05b24df8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:15.369632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:15.371121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:15.371594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:15.372077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] PATH_INFO: `/v3/users/2f8e2d51704d418085a6b37077f56a7e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:15.373759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] RBAC: Authorizing `identity:get_user(user_id=2f8e2d51704d418085a6b37077f56a7e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:15.376256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:15.379158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 170/330] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:15 2022] GET /identity/v3/users/2f8e2d51704d418085a6b37077f56a7e => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:15.385273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:15.428479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62881-dde8-49a8-b78e-bd790a2e06ce None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:15.447394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:15.481698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:15.482332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:15.482608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:15.482899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:15.484481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:15.484794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:15.486166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:15.501708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 161/331] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:15.504251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:15.513602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076ae51-2258-4c63-8fac-3b3a1a44c231 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:15.528599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:15.556343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:15.556992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:15.557272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:15.557546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:15.558362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] RBAC: Adding query filter params (role.id=471d5a03175b40dd8dc1c327195e2885, user.id=2f8e2d51704d418085a6b37077f56a7e, scope.system=all) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:15.558673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:15.560143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:15.573872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 171/332] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Aug 6 14:17:15 2022] GET /identity/v3/role_assignments?user.id=2f8e2d51704d418085a6b37077f56a7e&scope.system=all&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:15.642915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59f7ed-ad9f-4ced-9810-0a6ca862435c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:15.684858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8396afd-cff2-4b68-a18d-474d4b4c5eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:15.685879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8396afd-cff2-4b68-a18d-474d4b4c5eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:15.686374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8396afd-cff2-4b68-a18d-474d4b4c5eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:15.686893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8396afd-cff2-4b68-a18d-474d4b4c5eb0 None admin] PATH_INFO: `/v3/users/307ce9a52e08458b8d5126d98e1b9b57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:15.689316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8396afd-cff2-4b68-a18d-474d4b4c5eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=307ce9a52e08458b8d5126d98e1b9b57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:15.691036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8396afd-cff2-4b68-a18d-474d4b4c5eb0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:15.693285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 162/333] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:15 2022] GET /identity/v3/users/307ce9a52e08458b8d5126d98e1b9b57 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:15.699370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:15.820673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f67359-d41c-4a85-aa3a-d6870f985597 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:15.856655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482e2ec-1ade-47d5-8637-faeedc0a392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:15.857375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482e2ec-1ade-47d5-8637-faeedc0a392e None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:15.857656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482e2ec-1ade-47d5-8637-faeedc0a392e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:15.857932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482e2ec-1ade-47d5-8637-faeedc0a392e None admin] PATH_INFO: `/v3/system/users/307ce9a52e08458b8d5126d98e1b9b57/roles/135b77755ad64d97bfb82803f1c574f6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:15.859450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482e2ec-1ade-47d5-8637-faeedc0a392e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=307ce9a52e08458b8d5126d98e1b9b57, role_id=135b77755ad64d97bfb82803f1c574f6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:15.860828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482e2ec-1ade-47d5-8637-faeedc0a392e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:15.873257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 172/334] 10.222.0.17 () {58 vars in 1334 bytes} [Sat Aug 6 14:17:15 2022] PUT /identity/v3/system/users/307ce9a52e08458b8d5126d98e1b9b57/roles/135b77755ad64d97bfb82803f1c574f6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:17.872762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068f1d6-abe0-4f7f-b6ab-22e9a05e8ab0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:17.873705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068f1d6-abe0-4f7f-b6ab-22e9a05e8ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:17.874195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068f1d6-abe0-4f7f-b6ab-22e9a05e8ab0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:17.875765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 163/335] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:17.892400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388caafc-b7ab-43ea-b9b5-e973d99d3f28 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:17.892965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388caafc-b7ab-43ea-b9b5-e973d99d3f28 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:17.892965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388caafc-b7ab-43ea-b9b5-e973d99d3f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:17.920160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-388caafc-b7ab-43ea-b9b5-e973d99d3f28 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:17.921360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-388caafc-b7ab-43ea-b9b5-e973d99d3f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:17.956233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:18.004849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 173/336] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:18.041157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068f1d6-abe0-4f7f-b6ab-22e9a05e8ab0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:18.087678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4317-eb02-43a7-a6df-ed0724d941ba None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.088409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4317-eb02-43a7-a6df-ed0724d941ba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.088887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4317-eb02-43a7-a6df-ed0724d941ba None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.090706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 174/337] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:18.096612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9e8d-6d56-48ee-8e13-4bbd3562c898 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.097164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9e8d-6d56-48ee-8e13-4bbd3562c898 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.097641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9e8d-6d56-48ee-8e13-4bbd3562c898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.105685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:18.126697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ec7f9e8d-6d56-48ee-8e13-4bbd3562c898 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:18.128267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ec7f9e8d-6d56-48ee-8e13-4bbd3562c898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:18.226777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 175/338] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:18.231204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068f1d6-abe0-4f7f-b6ab-22e9a05e8ab0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:18.251247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f9e8d-6d56-48ee-8e13-4bbd3562c898 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:18.311262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:18.316625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.317133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.317609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.337859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:18.345134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:18.346001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:18.346831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:18.349871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 164/339] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:18.363996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:18.438905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:18.446333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f9e8d-6d56-48ee-8e13-4bbd3562c898 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:18.499746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:18.500763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.501246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.501701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.518131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:18.524080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:18.524996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:18.525791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:18.535256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 176/340] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:18.536419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:18.577410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201554c-68ee-4991-9025-e8233845f4e9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:18.587370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:18.630504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec22b9e-79c2-4dbf-bd12-0d19c4e4df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:18.631244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22b9e-79c2-4dbf-bd12-0d19c4e4df83 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.631532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22b9e-79c2-4dbf-bd12-0d19c4e4df83 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.631809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22b9e-79c2-4dbf-bd12-0d19c4e4df83 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.632528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22b9e-79c2-4dbf-bd12-0d19c4e4df83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:18.633382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22b9e-79c2-4dbf-bd12-0d19c4e4df83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:18.634766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22b9e-79c2-4dbf-bd12-0d19c4e4df83 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:18.644808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 165/341] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:18.650075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1619a28-0dfc-47a0-9659-e117981f5f4d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.650367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1619a28-0dfc-47a0-9659-e117981f5f4d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.650647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1619a28-0dfc-47a0-9659-e117981f5f4d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.652630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 166/342] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:18.653472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1619a28-0dfc-47a0-9659-e117981f5f4d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:18.741908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf165-9d1d-476e-8aa0-3aab997785dd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:18.801834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c523944-f1e3-4fe3-8208-cc0ecce0d675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:18.802893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c523944-f1e3-4fe3-8208-cc0ecce0d675 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.803379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c523944-f1e3-4fe3-8208-cc0ecce0d675 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.803867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c523944-f1e3-4fe3-8208-cc0ecce0d675 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.805313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c523944-f1e3-4fe3-8208-cc0ecce0d675 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:18.805820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c523944-f1e3-4fe3-8208-cc0ecce0d675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:18.815734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c523944-f1e3-4fe3-8208-cc0ecce0d675 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:18.830895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1619a28-0dfc-47a0-9659-e117981f5f4d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:18.831509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 177/343] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:18.833081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d240026-a9a9-411a-8281-175e0cdee6df None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.833575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d240026-a9a9-411a-8281-175e0cdee6df None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.834103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d240026-a9a9-411a-8281-175e0cdee6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.866909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4d240026-a9a9-411a-8281-175e0cdee6df None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:18.868169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4d240026-a9a9-411a-8281-175e0cdee6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:18.879799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad48218-08d6-4eb0-9354-40728c2b36ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:18.880433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad48218-08d6-4eb0-9354-40728c2b36ee None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:18.880767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad48218-08d6-4eb0-9354-40728c2b36ee None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:18.880990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad48218-08d6-4eb0-9354-40728c2b36ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:18.913099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:18.926287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:18.989277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad48218-08d6-4eb0-9354-40728c2b36ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:18.990734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad48218-08d6-4eb0-9354-40728c2b36ee None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:19.006287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 178/344] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:19.007254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d240026-a9a9-411a-8281-175e0cdee6df None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.119396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 167/345] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:19.121123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52531b23-c492-46b0-aef0-e24832a43509 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.121613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52531b23-c492-46b0-aef0-e24832a43509 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.122111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52531b23-c492-46b0-aef0-e24832a43509 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.124103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 168/346] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:19.125058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52531b23-c492-46b0-aef0-e24832a43509 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.155107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d240026-a9a9-411a-8281-175e0cdee6df None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:19.190519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a24bf-1922-4033-82a5-17b06dc5f206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:19.191626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a24bf-1922-4033-82a5-17b06dc5f206 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.192013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a24bf-1922-4033-82a5-17b06dc5f206 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.192597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a24bf-1922-4033-82a5-17b06dc5f206 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.209700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:19.215356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:19.254401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52531b23-c492-46b0-aef0-e24832a43509 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:19.274638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a24bf-1922-4033-82a5-17b06dc5f206 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:19.276502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a24bf-1922-4033-82a5-17b06dc5f206 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:19.296003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:19.296788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.297075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.297352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.312587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:19.317735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:19.318654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:19.319572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 169/347] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:19.320823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.400283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 179/348] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:19.401644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47877f-75a2-4721-ab1b-ea26ab8efd9c None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.401933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47877f-75a2-4721-ab1b-ea26ab8efd9c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.402208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47877f-75a2-4721-ab1b-ea26ab8efd9c None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.403684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 180/349] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:19.404551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ac7b5-55a0-4dad-947a-a5ad830dd87e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.404827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ac7b5-55a0-4dad-947a-a5ad830dd87e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.405096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ac7b5-55a0-4dad-947a-a5ad830dd87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.423497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-990ac7b5-55a0-4dad-947a-a5ad830dd87e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:19.424868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-990ac7b5-55a0-4dad-947a-a5ad830dd87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:19.444286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e611852-6a82-4acb-9b38-4e45fc312c66 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:19.472704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:19.478985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:19.484235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75e5e0-a2ee-410e-9903-eed572f901c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:19.484883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e5e0-a2ee-410e-9903-eed572f901c4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.485178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e5e0-a2ee-410e-9903-eed572f901c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.485455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e5e0-a2ee-410e-9903-eed572f901c4 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.486680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75e5e0-a2ee-410e-9903-eed572f901c4 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:19.488176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75e5e0-a2ee-410e-9903-eed572f901c4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:19.490976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 170/350] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:19.491235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a230a-2f49-470b-9b1f-5d738a62fb1d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.491516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a230a-2f49-470b-9b1f-5d738a62fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.491786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a230a-2f49-470b-9b1f-5d738a62fb1d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.493176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 171/351] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:19.493845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a230a-2f49-470b-9b1f-5d738a62fb1d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.521536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 181/352] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:19.522037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ac7b5-55a0-4dad-947a-a5ad830dd87e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.620853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a230a-2f49-470b-9b1f-5d738a62fb1d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:19.631100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ac7b5-55a0-4dad-947a-a5ad830dd87e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:19.661613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71ff63-e1b8-413c-b17f-b9072c0bc91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:19.662483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71ff63-e1b8-413c-b17f-b9072c0bc91c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.662765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71ff63-e1b8-413c-b17f-b9072c0bc91c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.663077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71ff63-e1b8-413c-b17f-b9072c0bc91c None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.663819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71ff63-e1b8-413c-b17f-b9072c0bc91c None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:19.664688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71ff63-e1b8-413c-b17f-b9072c0bc91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:19.666130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71ff63-e1b8-413c-b17f-b9072c0bc91c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:19.670240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:19.670934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.671223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.671503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.672671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:19.674173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:19.675517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 172/353] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:19.676319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 182/354] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:19.677026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136405b3-8f34-4de8-ad82-346ac50f3a2e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.677162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.677309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136405b3-8f34-4de8-ad82-346ac50f3a2e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.677591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136405b3-8f34-4de8-ad82-346ac50f3a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.701926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-136405b3-8f34-4de8-ad82-346ac50f3a2e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:19.703147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-136405b3-8f34-4de8-ad82-346ac50f3a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:19.733052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:19.769410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:19.787755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 173/355] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:19.789546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f6995-f89b-4ca1-bb91-d7917fd72bc0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.789827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f6995-f89b-4ca1-bb91-d7917fd72bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.790099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f6995-f89b-4ca1-bb91-d7917fd72bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.802972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48399d1d-855f-4a36-8084-55ac5bd032cd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:19.810953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1e6f6995-f89b-4ca1-bb91-d7917fd72bc0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:19.812877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1e6f6995-f89b-4ca1-bb91-d7917fd72bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:19.846357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:19.847043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:19.847325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:19.847597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:19.855542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:19.856992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:19.858804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 183/356] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:19.860698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.903194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 174/357] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:19.904621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f6995-f89b-4ca1-bb91-d7917fd72bc0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:19.981677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd68db-2de5-43c3-9c94-e5b11d11cb10 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:19.992127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.024662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.025452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.025736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.028148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.032835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f6995-f89b-4ca1-bb91-d7917fd72bc0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.038841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:20.044966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:20.045972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:20.046742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 184/358] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:20.047867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.057488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.083733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.084704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.085130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.085423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.167257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.167504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb9e79-e8cf-427d-b062-77d971e650fc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.168848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.207477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ec795-d348-4e91-b500-c27536633691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.208111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ec795-d348-4e91-b500-c27536633691 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.208392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ec795-d348-4e91-b500-c27536633691 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.208665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ec795-d348-4e91-b500-c27536633691 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.210419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ec795-d348-4e91-b500-c27536633691 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.212430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ec795-d348-4e91-b500-c27536633691 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.214055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 185/359] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:20.215002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ec795-d348-4e91-b500-c27536633691 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.255129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.301358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 175/360] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:20.302910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.324562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.334844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ec795-d348-4e91-b500-c27536633691 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.373873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.374577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.374885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.377687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.388049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:20.393232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:20.394131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:20.396018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 186/361] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:20.396900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.424864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5c58-1dd7-4a10-b09e-16d7f656ff86 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.465985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.466628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.466951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.467232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.468046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=6563532b47244425a9fbdda6e507bf2c) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:20.468355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.469782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.475111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 176/362] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:20.475887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.510016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.517309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6d41a-e2c0-448c-a50f-062a7bc3c77c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.555583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.556341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.556626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.556911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.568279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:20.573431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:20.574348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:20.575193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 187/363] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:20.575862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.580410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.604322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63283800-b217-459f-9c1e-a2b9802035d1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.642171 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.642927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.643288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.643643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.644505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:20.644885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.646331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.654732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 177/364] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:20.655469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.698079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef152b-1fef-41e1-9a02-d24157780976 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.738672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.739433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.739723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.740002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.740797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=6563532b47244425a9fbdda6e507bf2c) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:20.741112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.743250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.754914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 188/365] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:20.754914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.772407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88066a1b-95f5-4930-9f40-a3161f25f496 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.785294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.811130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.812409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.812409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.812409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.813638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.815125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.821596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 178/366] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:20.822547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.841610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:20.905314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91467-1a5e-47d3-a449-ca4e9fe744e4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.940539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36e27f-fbde-40f2-a078-671e3128f913 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:20.956036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.956668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.956947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.957218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.957933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:20.958238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.960059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.968974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 189/367] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:20.969701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:20.980223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:20.980970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:20.981329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:20.981672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:20.982501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:20.982906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:20.984431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:20.993815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 179/368] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:20.994771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.066368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:21.091764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22508cb-7d9c-4595-9a60-6731d70f9e54 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.105108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:21.130860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.131667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.131942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.132223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:21.141613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df426f7-fe55-40a8-a3d1-cf8a2cdfe63f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.192049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.192782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.193062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.193337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:21.194493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:21.196487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:21.198505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 180/369] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:21.199435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.215761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:21.218474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:21.306529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081698e-cb4b-4aa6-99d9-ce2f5040a255 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.323346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:21.347434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 190/370] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:21.348572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.353715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.355797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.356114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.356424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] PATH_INFO: `/v3/users/307ce9a52e08458b8d5126d98e1b9b57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:21.357828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] RBAC: Authorizing `identity:get_user(user_id=307ce9a52e08458b8d5126d98e1b9b57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:21.359747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:21.365882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 181/371] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:17:21 2022] GET /identity/v3/users/307ce9a52e08458b8d5126d98e1b9b57 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:21.365882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.382767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:21.480286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9d7c2-4622-4b2d-9f25-5872a1074805 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.497486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bc7c-67a4-4c1c-9a31-5a1303da271c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.527841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.528806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.529351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.529844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:21.538446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.539711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.539992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.540264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:21.541175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=923039d4dbc0458592a537e8bf17dd57) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:21.541458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:21.542897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:21.549289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 182/372] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:21 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:21.550244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.590287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:21.612097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:21.613927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:21.641581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:21.689117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768668e7-b71d-40ce-b203-5251ae766072 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.728944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 191/373] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:21.730109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.758849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.759475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.759808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.760088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:21.761357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:21.762906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:21.772178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 183/374] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:21 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:21.777544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.860226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:21.883706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427a2f-667c-418c-9228-0e28d451931b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.923604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc0d47-317a-468f-bbd6-24e1bc45f3e3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:21.949566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.951343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.951987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.953609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:21.954626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] RBAC: Adding query filter params (role.id=135b77755ad64d97bfb82803f1c574f6, user.id=307ce9a52e08458b8d5126d98e1b9b57, scope.system=all) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:21.955975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:21.957743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:21.983498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 192/375] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Aug 6 14:17:21 2022] GET /identity/v3/role_assignments?user.id=307ce9a52e08458b8d5126d98e1b9b57&scope.system=all&role.id=135b77755ad64d97bfb82803f1c574f6 => generated 510 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:21.990914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:21.994935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:21.998278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:21.998859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:21.999370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] PATH_INFO: `/v3/projects/3509ae96075e47edbbf7f243bb4b1186` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:22.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] RBAC: Authorizing `identity:get_project(project_id=3509ae96075e47edbbf7f243bb4b1186)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:22.004542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:22.006376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 184/376] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:21 2022] GET /identity/v3/projects/3509ae96075e47edbbf7f243bb4b1186 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:22.013628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:22.041492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:22.113641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36863a01-ba56-4123-baf4-ed0d802d7b7b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:22.153992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:22.155070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:22.155577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:22.156079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:22.157596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:22.159310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:22.161557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 193/377] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:21 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:22.167795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:22.182099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:22.248013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead5723-1ad0-4934-b786-2490b4292d37 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:22.286996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f6f27-3232-40bd-86a8-a6c515f9ec73 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:22.322121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:22.322792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:22.323088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:22.323349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] PATH_INFO: `/v3/groups/424dc58ad1834c0fb7d7b317b10def7a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:22.332699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c451f-230c-4ecc-a606-3f610a1c481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:22.333810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c451f-230c-4ecc-a606-3f610a1c481b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:22.334306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c451f-230c-4ecc-a606-3f610a1c481b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:22.334779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c451f-230c-4ecc-a606-3f610a1c481b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:22.335529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] RBAC: Authorizing `identity:get_group(group_id=424dc58ad1834c0fb7d7b317b10def7a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:22.336809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c451f-230c-4ecc-a606-3f610a1c481b None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=923039d4dbc0458592a537e8bf17dd57) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:22.337327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c451f-230c-4ecc-a606-3f610a1c481b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:22.337529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:22.339244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c451f-230c-4ecc-a606-3f610a1c481b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:22.339529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 185/378] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:22 2022] GET /identity/v3/groups/424dc58ad1834c0fb7d7b317b10def7a => generated 229 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:22.345179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:22.346817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 194/379] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:22 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:22.459965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:22.492710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a024b97-f84c-42f3-bf02-16e281f3c67c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:22.531061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155f2e0-8afb-41e2-a926-eafa8759603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:22.531689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155f2e0-8afb-41e2-a926-eafa8759603a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:22.531967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155f2e0-8afb-41e2-a926-eafa8759603a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:22.532241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155f2e0-8afb-41e2-a926-eafa8759603a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:22.533019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155f2e0-8afb-41e2-a926-eafa8759603a None admin] RBAC: Adding query filter params (group.id=424dc58ad1834c0fb7d7b317b10def7a, role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=3509ae96075e47edbbf7f243bb4b1186) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:22.533331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155f2e0-8afb-41e2-a926-eafa8759603a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:22.534759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155f2e0-8afb-41e2-a926-eafa8759603a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:22.543178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 186/380] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:22 2022] GET /identity/v3/role_assignments?group.id=424dc58ad1834c0fb7d7b317b10def7a&scope.project.id=3509ae96075e47edbbf7f243bb4b1186&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:22.781939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d503c3-3213-4996-8dbe-690699c0e7c6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:22.782250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d503c3-3213-4996-8dbe-690699c0e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:22.782425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d503c3-3213-4996-8dbe-690699c0e7c6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:22.784082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 195/381] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:22.789894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9ac80-4b6f-4b57-8dde-e6a98c856623 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:22.790288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9ac80-4b6f-4b57-8dde-e6a98c856623 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:22.790661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9ac80-4b6f-4b57-8dde-e6a98c856623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:22.814504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ffa9ac80-4b6f-4b57-8dde-e6a98c856623 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:22.815723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ffa9ac80-4b6f-4b57-8dde-e6a98c856623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:22.900331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 187/382] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:22.925279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d503c3-3213-4996-8dbe-690699c0e7c6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:23.010309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:23.052496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d503c3-3213-4996-8dbe-690699c0e7c6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:23.099770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b6789-3cf0-453f-9743-d552c86b1779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:23.100782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b6789-3cf0-453f-9743-d552c86b1779 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:23.101290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b6789-3cf0-453f-9743-d552c86b1779 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:23.102324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b6789-3cf0-453f-9743-d552c86b1779 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:23.119867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b6789-3cf0-453f-9743-d552c86b1779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-393b6789-3cf0-453f-9743-d552c86b1779 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:23.126270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:23.127867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:23.129286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:23.132078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 196/383] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:23.138303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9ac80-4b6f-4b57-8dde-e6a98c856623 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:23.200513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40878165-8e03-491a-84df-ab8b4850e4de None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:23.200859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40878165-8e03-491a-84df-ab8b4850e4de None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:23.201074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40878165-8e03-491a-84df-ab8b4850e4de None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:23.202559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 197/384] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:23.208724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787014-a0dc-4245-bd3f-c0d267ffb87e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:23.209013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787014-a0dc-4245-bd3f-c0d267ffb87e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:23.209300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787014-a0dc-4245-bd3f-c0d267ffb87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:23.245390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-88787014-a0dc-4245-bd3f-c0d267ffb87e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:23.246583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-88787014-a0dc-4245-bd3f-c0d267ffb87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:23.271683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:23.341579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 198/385] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:23.345224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9ac80-4b6f-4b57-8dde-e6a98c856623 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:23.386284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88787014-a0dc-4245-bd3f-c0d267ffb87e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:23.402916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:23.403711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:23.403990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:23.406124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:23.406903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:23.407218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:23.408586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:23.429293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 188/386] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:23.431540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:23.514921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:23.583156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88787014-a0dc-4245-bd3f-c0d267ffb87e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:23.623579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef48c65-d3ad-4456-bbd4-543b9e9efc38 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:23.653142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:23.653802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:23.654134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:23.654418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:23.674720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:23.680379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:23.681243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:23.682047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:23.683783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 199/387] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:23.688104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:23.688664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:23.705456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:23.706115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:23.706394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:23.706667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:23.712049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:23.713484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:23.715735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 189/388] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:23 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:23.720698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:23.874646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af931b-416f-4b30-917d-c82d95e93cdf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:23.909740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:23.910672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:23.911183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:23.911751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:23.913145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:23.914789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:23.917002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 190/389] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:23 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:23.918557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433194cb-286f-444f-96f7-b9c1651299f1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:23.926532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:23.973441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:24.015393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:24.033505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeb978-bd79-4fc0-a5c2-f7242ddef32b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:24.036158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:24.038455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.039226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.041128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.045134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:24.045704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:24.047348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:24.066262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 200/390] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:24.068281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:24.123597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1922288-bf34-4b58-89da-a8710cb39921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:24.124283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1922288-bf34-4b58-89da-a8710cb39921 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.124570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1922288-bf34-4b58-89da-a8710cb39921 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.124897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1922288-bf34-4b58-89da-a8710cb39921 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.131700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1922288-bf34-4b58-89da-a8710cb39921 None admin] RBAC: Authorizing `identity:create_grant(project_id=6563532b47244425a9fbdda6e507bf2c, group_id=05c35b6a9c5441e6a3d3bd22bd53fabb, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:24.133180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1922288-bf34-4b58-89da-a8710cb39921 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:24.162126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 191/391] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Aug 6 14:17:23 2022] PUT /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:17:24.185215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddd730-f4fa-4fbf-89c4-cf5fcc5929dd None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.185554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddd730-f4fa-4fbf-89c4-cf5fcc5929dd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.185805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddd730-f4fa-4fbf-89c4-cf5fcc5929dd None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.187247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 192/392] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:24.193586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d1484-b847-4972-9eb1-22f8867bf968 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:24.203308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc4c8d-c5ba-4235-b4cf-aa2efcf2f5f1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.203612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc4c8d-c5ba-4235-b4cf-aa2efcf2f5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.203872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc4c8d-c5ba-4235-b4cf-aa2efcf2f5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.237877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9edc4c8d-c5ba-4235-b4cf-aa2efcf2f5f1 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:24.239144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9edc4c8d-c5ba-4235-b4cf-aa2efcf2f5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:24.245110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:24.245777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.246060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.246337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.247660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:24.249138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:24.251212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 201/393] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:24 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:24.259997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:24.336363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 193/394] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:24.364047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:24.365217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc4c8d-c5ba-4235-b4cf-aa2efcf2f5f1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:24.433100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea1338-6aad-4390-9468-7600cc5bd029 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:24.451812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:24.504914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc4c8d-c5ba-4235-b4cf-aa2efcf2f5f1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:24.541486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:24.542150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.542424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.542694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.545920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:24.547597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:24.548697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:24.550439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 202/395] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:24 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:24.552099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.552777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.553221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.559868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:24.566321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:24.572963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:24.574015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:24.575015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:24.578052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 194/396] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:24.583321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:24.692244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f2bd5-23e2-49bb-a617-07590fcac2c7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:24.719118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:24.729131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:24.729756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.730079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.730287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.731024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:24.731324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:24.732796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:24.742931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 195/397] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:24.749129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:24.795357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7b34e-dc09-45f8-be10-622f7f2804cd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:24.839586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:24.874752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11012e8b-a723-45d3-8972-66ec3fee2441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:24.875479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11012e8b-a723-45d3-8972-66ec3fee2441 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.875803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11012e8b-a723-45d3-8972-66ec3fee2441 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.876082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11012e8b-a723-45d3-8972-66ec3fee2441 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.878303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11012e8b-a723-45d3-8972-66ec3fee2441 None admin] RBAC: Authorizing `identity:create_grant(project_id=6563532b47244425a9fbdda6e507bf2c, group_id=05c35b6a9c5441e6a3d3bd22bd53fabb, role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:24.879836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11012e8b-a723-45d3-8972-66ec3fee2441 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:24.898662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 203/398] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Aug 6 14:17:24 2022] PUT /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:24.900008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfc254-cfda-4c90-b95c-9ea8c7c6dfb3 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.900292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfc254-cfda-4c90-b95c-9ea8c7c6dfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.900567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfc254-cfda-4c90-b95c-9ea8c7c6dfb3 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.907442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 204/399] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:24.909260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea2a56-f8ba-40b9-be84-1c3d56568245 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:24.909549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea2a56-f8ba-40b9-be84-1c3d56568245 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:24.909825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea2a56-f8ba-40b9-be84-1c3d56568245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:24.931850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c0061-0ccc-430b-89ed-e2cce0b15f27 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:24.936525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bbea2a56-f8ba-40b9-be84-1c3d56568245 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:24.936525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bbea2a56-f8ba-40b9-be84-1c3d56568245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:25.002964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.003967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.004440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.004922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.007960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:25.009842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:25.012193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 196/400] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:24 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:25.020070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.021713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 205/401] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:25.049233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea2a56-f8ba-40b9-be84-1c3d56568245 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.112579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:25.176315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea2a56-f8ba-40b9-be84-1c3d56568245 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:25.190021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:25.221266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.221912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.222180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.222518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.222781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ba853-5c11-4fe9-ade7-9644c2d7b563 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:25.235818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:25.241281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:25.242363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:25.243209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:25.244134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 206/402] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:25.250127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.268650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e25fe-0a95-4cc8-9008-bb11974d8411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.269566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e25fe-0a95-4cc8-9008-bb11974d8411 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.270207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e25fe-0a95-4cc8-9008-bb11974d8411 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.271053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e25fe-0a95-4cc8-9008-bb11974d8411 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.272422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e25fe-0a95-4cc8-9008-bb11974d8411 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:25.275437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e25fe-0a95-4cc8-9008-bb11974d8411 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:25.278023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 197/403] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:25 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:25.279703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6783102-31a9-40b9-b41c-87032fc255b6 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.280155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6783102-31a9-40b9-b41c-87032fc255b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.280598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6783102-31a9-40b9-b41c-87032fc255b6 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.281949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 198/404] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:25.283498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6783102-31a9-40b9-b41c-87032fc255b6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.359533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca6539-de73-4cd1-8f0f-4440d009cd02 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:25.400244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad03746-d6d0-4a9e-be20-0ff35103c78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.401022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad03746-d6d0-4a9e-be20-0ff35103c78e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.401367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad03746-d6d0-4a9e-be20-0ff35103c78e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.401709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad03746-d6d0-4a9e-be20-0ff35103c78e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.402689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad03746-d6d0-4a9e-be20-0ff35103c78e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:25.403413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad03746-d6d0-4a9e-be20-0ff35103c78e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:25.405209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad03746-d6d0-4a9e-be20-0ff35103c78e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:25.414055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6783102-31a9-40b9-b41c-87032fc255b6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:25.415011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 207/405] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:25.421121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef4a0-ba2a-4f7f-9e63-8b42b67b3f53 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.423334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef4a0-ba2a-4f7f-9e63-8b42b67b3f53 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.423696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef4a0-ba2a-4f7f-9e63-8b42b67b3f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.435668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:25.451876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-30fef4a0-ba2a-4f7f-9e63-8b42b67b3f53 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:25.453061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-30fef4a0-ba2a-4f7f-9e63-8b42b67b3f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:25.466446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.467655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.468156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.469007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.471929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] RBAC: Authorizing `identity:create_grant(project_id=923039d4dbc0458592a537e8bf17dd57, group_id=05c35b6a9c5441e6a3d3bd22bd53fabb, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:25.473593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:25.494185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 199/406] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Aug 6 14:17:25 2022] PUT /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:25.500907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:25.500991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.576019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 208/407] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:25.619983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fef4a0-ba2a-4f7f-9e63-8b42b67b3f53 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.650770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95227e24-059d-45a1-a1dd-0a79de69b735 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:25.688665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.689293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.689571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.689844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.691026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:25.692446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:25.694858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 200/408] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:25 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:25.702281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.760370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:25.771524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fef4a0-ba2a-4f7f-9e63-8b42b67b3f53 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:25.829796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aad528-e510-4fad-9962-d96b04e4ba6b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:25.831225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.832145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.832661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.833104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.866947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:25.868212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:25.868596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:25.868973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:25.870418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:25.871993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:25.874055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 201/409] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:25 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:25.879316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:25.879990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:25.884389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:25.885264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:25.886054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 209/410] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:25.895837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:25.973644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:25.991269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ad0f0-546e-49f9-be43-09d48ad5cb32 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:26.028021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:26.028729 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.029260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.029540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.031810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] RBAC: Authorizing `identity:create_grant(project_id=923039d4dbc0458592a537e8bf17dd57, group_id=05c35b6a9c5441e6a3d3bd22bd53fabb, role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:26.034178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:26.038322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:26.050320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 202/411] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Aug 6 14:17:25 2022] PUT /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57/groups/05c35b6a9c5441e6a3d3bd22bd53fabb/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:26.145773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3224c6-39a9-47f6-80c2-90a57b31fd5d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:26.195986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:26.197021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.197493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.197979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.198997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:26.199591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:26.202538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:26.213183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 210/412] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:26.220140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:26.331246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da80c2f-6af1-4a87-82af-9f4fceb4380f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:26.365798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ebc41-e45f-4832-ba23-7cda4a062b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:26.366444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ebc41-e45f-4832-ba23-7cda4a062b0a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.366721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ebc41-e45f-4832-ba23-7cda4a062b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.367539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ebc41-e45f-4832-ba23-7cda4a062b0a None admin] PATH_INFO: `/v3/groups/424dc58ad1834c0fb7d7b317b10def7a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.368691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ebc41-e45f-4832-ba23-7cda4a062b0a None admin] RBAC: Authorizing `identity:get_group(group_id=424dc58ad1834c0fb7d7b317b10def7a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:26.370108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ebc41-e45f-4832-ba23-7cda4a062b0a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:26.372029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 203/413] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:26 2022] GET /identity/v3/groups/424dc58ad1834c0fb7d7b317b10def7a => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:26.377476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:26.466462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:26.494332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a68c0c-2b33-46c4-8b7c-acabfee5f795 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:26.508474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007380e8-b15d-425d-baeb-c9b9fa50f8b6 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.508789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007380e8-b15d-425d-baeb-c9b9fa50f8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.509027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007380e8-b15d-425d-baeb-c9b9fa50f8b6 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.510733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 204/414] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:26.516583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f153-507a-497f-bcd5-720cd6165b50 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.516870 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f153-507a-497f-bcd5-720cd6165b50 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.517141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f153-507a-497f-bcd5-720cd6165b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.534517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:26.535203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.535482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.535757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] PATH_INFO: `/v3/projects/3509ae96075e47edbbf7f243bb4b1186` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.536943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] RBAC: Authorizing `identity:get_project(project_id=3509ae96075e47edbbf7f243bb4b1186)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:26.538386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:26.541194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 211/415] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:26 2022] GET /identity/v3/projects/3509ae96075e47edbbf7f243bb4b1186 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:26.546657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:26.552026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0986f153-507a-497f-bcd5-720cd6165b50 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:26.554035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0986f153-507a-497f-bcd5-720cd6165b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:26.619244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:26.685418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 205/416] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:26.712600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b8dcd-af4c-477a-ac8c-2100ff29bd80 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:26.725451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986f153-507a-497f-bcd5-720cd6165b50 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:26.777163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:26.778399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.779164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.779629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] PATH_INFO: `/v3/projects/3509ae96075e47edbbf7f243bb4b1186/groups/424dc58ad1834c0fb7d7b317b10def7a/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.787700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] RBAC: Authorizing `identity:create_grant(project_id=3509ae96075e47edbbf7f243bb4b1186, group_id=424dc58ad1834c0fb7d7b317b10def7a, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:26.791453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:26.815822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 212/417] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Aug 6 14:17:26 2022] PUT /identity/v3/projects/3509ae96075e47edbbf7f243bb4b1186/groups/424dc58ad1834c0fb7d7b317b10def7a/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:26.888949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986f153-507a-497f-bcd5-720cd6165b50 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:26.957713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:26.958758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:26.959278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:26.959755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:26.983712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:26.989183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:26.990147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:26.991007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:26.993924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 206/418] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:27.004059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:27.035253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:27.119294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2145a-edc1-4250-804b-1cef3ea1f1ff None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:27.155792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690e2de-26e3-4100-a879-63ababc5e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:27.156416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690e2de-26e3-4100-a879-63ababc5e5ef None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:27.156695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690e2de-26e3-4100-a879-63ababc5e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:27.156967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690e2de-26e3-4100-a879-63ababc5e5ef None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:27.157676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690e2de-26e3-4100-a879-63ababc5e5ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:27.157977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690e2de-26e3-4100-a879-63ababc5e5ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:27.159399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690e2de-26e3-4100-a879-63ababc5e5ef None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:27.168818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 213/419] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:27.174242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:27.243173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:27.353261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3859b-9c09-410d-bd1e-2cf1c9f5a25a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:27.419303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea83610-27f7-4710-86c8-1b294e960f25 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:27.420278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea83610-27f7-4710-86c8-1b294e960f25 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:27.420766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea83610-27f7-4710-86c8-1b294e960f25 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:27.422604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 214/420] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:27.427880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0af0aa-8961-47bd-8169-94b7747275ab None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:27.428006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:27.428535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0af0aa-8961-47bd-8169-94b7747275ab None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:27.428647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:27.428868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:27.429147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:27.429548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0af0aa-8961-47bd-8169-94b7747275ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:27.463338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2a0af0aa-8961-47bd-8169-94b7747275ab None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:27.464881 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2a0af0aa-8961-47bd-8169-94b7747275ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:27.543002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:27.544363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:27.551992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 215/421] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:27.584558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:27.592921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0af0aa-8961-47bd-8169-94b7747275ab None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:27.599977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:27.675795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 207/422] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:27.681678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:27.794050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6fa58-590c-455c-9866-8d0fdacc5dcd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:27.824178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:27.834843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:27.835523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:27.836697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:27.836973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:27.838133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:27.840122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:27.842060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 208/423] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:27 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:27.847681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:27.881446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0af0aa-8961-47bd-8169-94b7747275ab None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:27.961432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b695134-a319-4150-901b-13f6e9eee7e6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:27.962515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:27.963308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:27.963821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:27.964300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:27.991770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:27.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:27.998565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:27.999374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:28.006917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 216/424] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:28.008782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:28.009589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.010014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.010442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.011755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:28.012978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.023520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:28.025688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 209/425] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:27 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:28.033911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.113299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:28.133525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:28.157783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ae3ba-b38a-4e14-adc6-7139c3ebf296 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:28.183249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fd0ba-5b25-45f5-b5fb-dfdc80630593 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:28.204271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42999c02-28a5-465e-bd9f-07c660dd3f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:28.205169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42999c02-28a5-465e-bd9f-07c660dd3f1e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.205738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42999c02-28a5-465e-bd9f-07c660dd3f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.206158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42999c02-28a5-465e-bd9f-07c660dd3f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.207325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42999c02-28a5-465e-bd9f-07c660dd3f1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:28.207790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42999c02-28a5-465e-bd9f-07c660dd3f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:28.209330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42999c02-28a5-465e-bd9f-07c660dd3f1e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:28.222105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 217/426] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:28.223698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cb2ef-d10e-40d4-98fc-239a37a9c527 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.224131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cb2ef-d10e-40d4-98fc-239a37a9c527 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.224537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5023f1-f359-4d04-9be5-6d7f418a0234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:28.224717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cb2ef-d10e-40d4-98fc-239a37a9c527 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.225543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5023f1-f359-4d04-9be5-6d7f418a0234 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.226040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5023f1-f359-4d04-9be5-6d7f418a0234 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.226733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 218/427] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:28.227199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5023f1-f359-4d04-9be5-6d7f418a0234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.228000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cb2ef-d10e-40d4-98fc-239a37a9c527 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.228664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5023f1-f359-4d04-9be5-6d7f418a0234 None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=6563532b47244425a9fbdda6e507bf2c) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:28.229226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5023f1-f359-4d04-9be5-6d7f418a0234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:28.230998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5023f1-f359-4d04-9be5-6d7f418a0234 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:28.238116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 210/428] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:28.239801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518deabf-97ba-4664-9853-50b615b1c94c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.240278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518deabf-97ba-4664-9853-50b615b1c94c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.240783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518deabf-97ba-4664-9853-50b615b1c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.270952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-518deabf-97ba-4664-9853-50b615b1c94c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:28.274857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-518deabf-97ba-4664-9853-50b615b1c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:28.414345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 211/429] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:28.437904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518deabf-97ba-4664-9853-50b615b1c94c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.444107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cb2ef-d10e-40d4-98fc-239a37a9c527 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:28.475934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:28.487282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defa6b7-4028-47ee-9a2b-cdaf5711bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:28.487910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defa6b7-4028-47ee-9a2b-cdaf5711bb4a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.488191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defa6b7-4028-47ee-9a2b-cdaf5711bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.488467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defa6b7-4028-47ee-9a2b-cdaf5711bb4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.507285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:28.552445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518deabf-97ba-4664-9853-50b615b1c94c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:28.560074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defa6b7-4028-47ee-9a2b-cdaf5711bb4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:28.561623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defa6b7-4028-47ee-9a2b-cdaf5711bb4a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:28.590057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d6250-1682-40bb-af12-a4c680704dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:28.591093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d6250-1682-40bb-af12-a4c680704dcb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.591581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d6250-1682-40bb-af12-a4c680704dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.592041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d6250-1682-40bb-af12-a4c680704dcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.603459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d6250-1682-40bb-af12-a4c680704dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7b1d6250-1682-40bb-af12-a4c680704dcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:28.608770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:28.609630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 14:17:28.610433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:28.613107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 212/430] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:28.620627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f765bf-27f4-4810-93a1-9b2f8bc54b32 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.621132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f765bf-27f4-4810-93a1-9b2f8bc54b32 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.621412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f765bf-27f4-4810-93a1-9b2f8bc54b32 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.622860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 213/431] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:28.623451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f765bf-27f4-4810-93a1-9b2f8bc54b32 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.681107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 219/432] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:28.682661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712246-fd4b-4813-bc6f-a73e8bc8a5e6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.682980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712246-fd4b-4813-bc6f-a73e8bc8a5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.683257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712246-fd4b-4813-bc6f-a73e8bc8a5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.703349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-fc712246-fd4b-4813-bc6f-a73e8bc8a5e6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:28.704530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fc712246-fd4b-4813-bc6f-a73e8bc8a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:28.734097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:28.779690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f765bf-27f4-4810-93a1-9b2f8bc54b32 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:28.803825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 220/433] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:28.804169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:28.807315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712246-fd4b-4813-bc6f-a73e8bc8a5e6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.850058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:28.850058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.850058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.850058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.850405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:28.850405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:28.850702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:28.865506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 214/434] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:28.866397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.938000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712246-fd4b-4813-bc6f-a73e8bc8a5e6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:28.980852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:28.981498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:28.981786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:28.982124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] PATH_INFO: `/v3/projects/6563532b47244425a9fbdda6e507bf2c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:28.983359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] RBAC: Authorizing `identity:get_project(project_id=6563532b47244425a9fbdda6e507bf2c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:28.984809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:28.986462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 221/435] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/projects/6563532b47244425a9fbdda6e507bf2c => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:28.987238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:28.997261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a6991-aada-4016-b309-83de68daf2a8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.012732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.035259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.035901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.036181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.036453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.048092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:29.053297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:29.054241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 215/436] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:29.055583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.098627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc41db0-8a6a-4263-8160-7e133c74e679 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.102049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.137007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.137631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.137912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.138189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.165206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa83be-7c27-41c4-8f7f-a0aca19336b0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.201690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.202598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.203096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.203542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.204849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:29.206463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:29.207652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:29.209550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:29.211523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 216/437] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:29.213593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.233725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.313338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a389b-a27c-400c-8376-28eaae703634 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.314578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 222/438] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:29.315453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.322691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.347813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5cb8b-4246-475d-bb71-4ca0e5d9dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.348430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5cb8b-4246-475d-bb71-4ca0e5d9dc43 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.348712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5cb8b-4246-475d-bb71-4ca0e5d9dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.348995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5cb8b-4246-475d-bb71-4ca0e5d9dc43 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.349706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5cb8b-4246-475d-bb71-4ca0e5d9dc43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:29.350007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5cb8b-4246-475d-bb71-4ca0e5d9dc43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:29.351948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5cb8b-4246-475d-bb71-4ca0e5d9dc43 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:29.359627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 217/439] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:29.360852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dadc7-7a40-4403-b6c4-9bea32ad3d10 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.361144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dadc7-7a40-4403-b6c4-9bea32ad3d10 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.361418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dadc7-7a40-4403-b6c4-9bea32ad3d10 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.362614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 218/440] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:29.363345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dadc7-7a40-4403-b6c4-9bea32ad3d10 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.428888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6e46b-e718-4719-9f94-aa53191d548d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.455424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.469856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.470644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.470952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.471229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.472077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=6563532b47244425a9fbdda6e507bf2c) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:29.472393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:29.473969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:29.477286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dadc7-7a40-4403-b6c4-9bea32ad3d10 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.479853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 223/441] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=6563532b47244425a9fbdda6e507bf2c&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:29.480702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.518926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886029b-0661-4f0f-9f6b-3b6294cc20ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.519927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886029b-0661-4f0f-9f6b-3b6294cc20ac None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.520379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886029b-0661-4f0f-9f6b-3b6294cc20ac None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.520816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886029b-0661-4f0f-9f6b-3b6294cc20ac None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.522172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886029b-0661-4f0f-9f6b-3b6294cc20ac None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:29.524369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886029b-0661-4f0f-9f6b-3b6294cc20ac None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:29.526314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 219/442] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:29.527319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10d5ce-c9eb-46e4-b102-cc2ac10e99a4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.527603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10d5ce-c9eb-46e4-b102-cc2ac10e99a4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.527876 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10d5ce-c9eb-46e4-b102-cc2ac10e99a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.546920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-be10d5ce-c9eb-46e4-b102-cc2ac10e99a4 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:29.548161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-be10d5ce-c9eb-46e4-b102-cc2ac10e99a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:29.592833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.614427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770fbc8-f743-4efd-ad2b-cbef4252e88d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.649442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 220/443] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:29.650891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10d5ce-c9eb-46e4-b102-cc2ac10e99a4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.666353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.667042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.667321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.671589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.749204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.767745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:29.771706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:29.813337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10d5ce-c9eb-46e4-b102-cc2ac10e99a4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.855729 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:29.856160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:29.856442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:29.858588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:29.859752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:29.860364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:29.862221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 221/444] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:29.862730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.888754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 224/445] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:29.889915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:29.902929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.967684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3fdf0-fca6-49f0-8417-fc8376fff994 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:29.973765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:29.985963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50982648-cbd1-47c0-9996-25482a000aad None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.002291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.002881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.003170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.003446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.013934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:30.018936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:30.019992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:30.020781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:30.021384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.021384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.021528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 222/446] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:30.021599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.021599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.021599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=471d5a03175b40dd8dc1c327195e2885, scope.project.id=923039d4dbc0458592a537e8bf17dd57) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:30.021599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.022853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.025872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.035216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 225/447] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:29 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=471d5a03175b40dd8dc1c327195e2885 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:30.035216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.134838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:30.141318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2772aa-3e23-4778-9606-9b2f744ac103 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.141855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38cd3-6b2d-4390-ad0d-c365f5a82daf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.180082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.180711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.180995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.181271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.181734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.182000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:30.182282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.182377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.182642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.182938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] PATH_INFO: `/v3/projects/923039d4dbc0458592a537e8bf17dd57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.183822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.184117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] RBAC: Authorizing `identity:get_project(project_id=923039d4dbc0458592a537e8bf17dd57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.185541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.187246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 223/448] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/projects/923039d4dbc0458592a537e8bf17dd57 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:30.192805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.192951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 226/449] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:30.198348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.216008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:30.312423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb06c8-c565-4d88-9f0f-cf9c9502cc3c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.332768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966431-0579-4b57-93e1-5c80bdb8caec None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.353474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.354126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.354405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.354706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] PATH_INFO: `/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.356012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] RBAC: Authorizing `identity:get_group(group_id=05c35b6a9c5441e6a3d3bd22bd53fabb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.357431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.364107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 224/450] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/groups/05c35b6a9c5441e6a3d3bd22bd53fabb => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:30.364901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.373870 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.374941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.376212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.376212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.402289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:30.445244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.446695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.461701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:30.473335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0e26b-f038-4aa4-b689-fdc57d4ea846 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.508104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.508719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.508999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.509275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.510053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] RBAC: Adding query filter params (group.id=05c35b6a9c5441e6a3d3bd22bd53fabb, role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=923039d4dbc0458592a537e8bf17dd57) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:30.510360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.511829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.517507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 225/451] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/role_assignments?group.id=05c35b6a9c5441e6a3d3bd22bd53fabb&scope.project.id=923039d4dbc0458592a537e8bf17dd57&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:30.545973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 227/452] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 14:17:30.552045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.650095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:30.656508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cb8ab-5de9-49b7-a0b9-b36a42e541d5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.690589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.691261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.691543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.691819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] PATH_INFO: `/v3/projects/3509ae96075e47edbbf7f243bb4b1186` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.692981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3509ae96075e47edbbf7f243bb4b1186)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.694408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.696116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 226/453] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/projects/3509ae96075e47edbbf7f243bb4b1186 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:30.701646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.801109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170e7da-9210-4793-b847-aa85ef59d1ed None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.815483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:30.838755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea258e0b-6579-4648-b9a4-dc2e647a4830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.839471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea258e0b-6579-4648-b9a4-dc2e647a4830 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.839753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea258e0b-6579-4648-b9a4-dc2e647a4830 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.840032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea258e0b-6579-4648-b9a4-dc2e647a4830 None admin] PATH_INFO: `/v3/groups/424dc58ad1834c0fb7d7b317b10def7a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea258e0b-6579-4648-b9a4-dc2e647a4830 None admin] RBAC: Authorizing `identity:get_group(group_id=424dc58ad1834c0fb7d7b317b10def7a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.842649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea258e0b-6579-4648-b9a4-dc2e647a4830 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.844762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 228/454] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/groups/424dc58ad1834c0fb7d7b317b10def7a => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:30.850566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:30.946411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcc0da-1bc6-4c0c-a00f-3730744205d4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:30.978309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eb17d-f119-4e0b-9f3f-f359562e4d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:30.978963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eb17d-f119-4e0b-9f3f-f359562e4d7a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:30.979239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eb17d-f119-4e0b-9f3f-f359562e4d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:30.979509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eb17d-f119-4e0b-9f3f-f359562e4d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:30.980282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eb17d-f119-4e0b-9f3f-f359562e4d7a None admin] RBAC: Adding query filter params (group.id=424dc58ad1834c0fb7d7b317b10def7a, role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=3509ae96075e47edbbf7f243bb4b1186) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:30.980589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eb17d-f119-4e0b-9f3f-f359562e4d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:30.981988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eb17d-f119-4e0b-9f3f-f359562e4d7a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:30.987557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 227/455] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Aug 6 14:17:30 2022] GET /identity/v3/role_assignments?group.id=424dc58ad1834c0fb7d7b317b10def7a&scope.project.id=3509ae96075e47edbbf7f243bb4b1186&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:33.526032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab46224-fb87-4934-b5af-4e855d8a660b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:33.526589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab46224-fb87-4934-b5af-4e855d8a660b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:33.526589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab46224-fb87-4934-b5af-4e855d8a660b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:33.528230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 229/456] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:33.533225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b32d7-98db-48ca-894b-380bc40ab376 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:33.533621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b32d7-98db-48ca-894b-380bc40ab376 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:33.533797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b32d7-98db-48ca-894b-380bc40ab376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:33.555856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4c9b32d7-98db-48ca-894b-380bc40ab376 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:33.557055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4c9b32d7-98db-48ca-894b-380bc40ab376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:33.562646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa08f44-3697-4bfb-b3e6-f8fa2c7d494f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:33.563227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa08f44-3697-4bfb-b3e6-f8fa2c7d494f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:33.563705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa08f44-3697-4bfb-b3e6-f8fa2c7d494f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:33.565545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 230/457] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:33.570598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809780f-a94c-413a-8843-9a71caa1194d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:33.571134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809780f-a94c-413a-8843-9a71caa1194d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:33.571613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809780f-a94c-413a-8843-9a71caa1194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:33.582410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:33.610258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a809780f-a94c-413a-8843-9a71caa1194d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:33.611509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a809780f-a94c-413a-8843-9a71caa1194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:33.669487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 228/458] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:33.697283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b32d7-98db-48ca-894b-380bc40ab376 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:33.745878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 231/459] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:33.822796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809780f-a94c-413a-8843-9a71caa1194d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:33.858888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b32d7-98db-48ca-894b-380bc40ab376 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:33.919021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bc43f-a075-4fdf-8bc6-4aadfad04879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:33.919677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bc43f-a075-4fdf-8bc6-4aadfad04879 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:33.919958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bc43f-a075-4fdf-8bc6-4aadfad04879 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:33.920231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bc43f-a075-4fdf-8bc6-4aadfad04879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:33.934484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bc43f-a075-4fdf-8bc6-4aadfad04879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:33.935969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bc43f-a075-4fdf-8bc6-4aadfad04879 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-173bc43f-a075-4fdf-8bc6-4aadfad04879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:33.959927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:33.961438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:33.962268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:33.962268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:33.962268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:33.962268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:33.963350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 229/460] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:33.964889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdd760-d28e-4305-b70b-2acd3fac1797 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:33.965169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdd760-d28e-4305-b70b-2acd3fac1797 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:33.965444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdd760-d28e-4305-b70b-2acd3fac1797 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:33.967238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 230/461] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:33.972326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdd760-d28e-4305-b70b-2acd3fac1797 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:33.986600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809780f-a94c-413a-8843-9a71caa1194d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:33.987953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.004832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.032220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b68006-36ec-42bf-b5ea-3d45f938dbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.032944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68006-36ec-42bf-b5ea-3d45f938dbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.033228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68006-36ec-42bf-b5ea-3d45f938dbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.033504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68006-36ec-42bf-b5ea-3d45f938dbd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.047040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b68006-36ec-42bf-b5ea-3d45f938dbd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.048761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b68006-36ec-42bf-b5ea-3d45f938dbd3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-a4b68006-36ec-42bf-b5ea-3d45f938dbd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:34.068272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:34.069172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:34.070537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:34.070537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:34.070537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:34.070537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:34.070537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 232/462] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:34.071518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e51e2-d7c7-4957-a280-f6864136425e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.071799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e51e2-d7c7-4957-a280-f6864136425e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.072071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e51e2-d7c7-4957-a280-f6864136425e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.091858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdd760-d28e-4305-b70b-2acd3fac1797 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.096028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dd0e51e2-d7c7-4957-a280-f6864136425e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:34.097209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dd0e51e2-d7c7-4957-a280-f6864136425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:34.130656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.131647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.132154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.133137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.134085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:34.134597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.136205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.150427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 231/463] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:34.151620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:34.180692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 233/464] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:34.181506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e51e2-d7c7-4957-a280-f6864136425e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:34.227990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.265193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35f366-edd9-4035-80ed-979645d2b6a5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.291786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.299164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e51e2-d7c7-4957-a280-f6864136425e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.302941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c8d5f-afa0-4b44-be25-4eb6c16ec6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.303624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c8d5f-afa0-4b44-be25-4eb6c16ec6de None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.303908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c8d5f-afa0-4b44-be25-4eb6c16ec6de None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.304877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c8d5f-afa0-4b44-be25-4eb6c16ec6de None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.305647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c8d5f-afa0-4b44-be25-4eb6c16ec6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:34.305955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c8d5f-afa0-4b44-be25-4eb6c16ec6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.307350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c8d5f-afa0-4b44-be25-4eb6c16ec6de None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.324296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 232/465] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:34.324296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc85b2-cb8c-48d9-abe7-0af55aa3ae51 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.324296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc85b2-cb8c-48d9-abe7-0af55aa3ae51 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.324296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc85b2-cb8c-48d9-abe7-0af55aa3ae51 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.324296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 233/466] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:34.324296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc85b2-cb8c-48d9-abe7-0af55aa3ae51 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:34.341772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.342449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.342794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.343106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.343783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.345216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.374908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 234/467] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:34 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:34.374908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:34.440811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc85b2-cb8c-48d9-abe7-0af55aa3ae51 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.473803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.481618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0f4f0-d7bf-484d-b6a1-34969fd245c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.482400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0f4f0-d7bf-484d-b6a1-34969fd245c6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.482684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0f4f0-d7bf-484d-b6a1-34969fd245c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.483049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0f4f0-d7bf-484d-b6a1-34969fd245c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.496416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0f4f0-d7bf-484d-b6a1-34969fd245c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.498049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0f4f0-d7bf-484d-b6a1-34969fd245c6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-22b0f4f0-d7bf-484d-b6a1-34969fd245c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:34.519168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:34.520097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:34.520919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:34.520919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:34.520919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:34.520919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:34.521773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce45c2-db44-47c7-aefe-b0187066be25 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.525826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 234/468] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:34.527977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09d25b-d8d1-42ef-bb5b-724a1a58d6ef None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.528268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09d25b-d8d1-42ef-bb5b-724a1a58d6ef None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.528545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09d25b-d8d1-42ef-bb5b-724a1a58d6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.547698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7f09d25b-d8d1-42ef-bb5b-724a1a58d6ef None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:34.548960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7f09d25b-d8d1-42ef-bb5b-724a1a58d6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:34.571462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.572399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.572681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.572951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.573633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.575090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.601802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 235/469] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:34 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:34.602369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:34.626759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.628074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 235/470] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:34.657541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09d25b-d8d1-42ef-bb5b-724a1a58d6ef None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:34.759969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20133f37-c11e-43c1-8c90-cc4547bb7fe0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.800233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f392a-08db-47d0-afd4-7ee81dc5486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.800862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f392a-08db-47d0-afd4-7ee81dc5486f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.801141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f392a-08db-47d0-afd4-7ee81dc5486f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.801416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f392a-08db-47d0-afd4-7ee81dc5486f None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.802142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f392a-08db-47d0-afd4-7ee81dc5486f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:34.802441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f392a-08db-47d0-afd4-7ee81dc5486f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.803837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f392a-08db-47d0-afd4-7ee81dc5486f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.811412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09d25b-d8d1-42ef-bb5b-724a1a58d6ef None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.821090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 236/471] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:34.823111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99cc7c-ff55-45b4-8d02-a64dbd31b745 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.823489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99cc7c-ff55-45b4-8d02-a64dbd31b745 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.823952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99cc7c-ff55-45b4-8d02-a64dbd31b745 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.825393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 237/472] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:34.826071 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.827698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99cc7c-ff55-45b4-8d02-a64dbd31b745 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:34.856505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e40d9e-3186-435e-b679-c85269fd1319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.857364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40d9e-3186-435e-b679-c85269fd1319 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.857364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40d9e-3186-435e-b679-c85269fd1319 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.857624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40d9e-3186-435e-b679-c85269fd1319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.869914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e40d9e-3186-435e-b679-c85269fd1319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.871497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e40d9e-3186-435e-b679-c85269fd1319 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-64e40d9e-3186-435e-b679-c85269fd1319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:34.891404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:34.892447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:34.893418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:34.893418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:34.893418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:34.893418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:34.893418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 236/473] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:34.894150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7639c71-930a-4871-87bb-f71cbf4e29e3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.894436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7639c71-930a-4871-87bb-f71cbf4e29e3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.894711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7639c71-930a-4871-87bb-f71cbf4e29e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.911443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:34.917955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a7639c71-930a-4871-87bb-f71cbf4e29e3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:34.919348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a7639c71-930a-4871-87bb-f71cbf4e29e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:34.953538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99cc7c-ff55-45b4-8d02-a64dbd31b745 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:34.988475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:34.989096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:34.989385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:34.989646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:34.990322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:34.992350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:34.996847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 237/474] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:34.998001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7639c71-930a-4871-87bb-f71cbf4e29e3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:35.034411 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 238/475] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:34 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:35.035150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:35.106740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:35.111806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7639c71-930a-4871-87bb-f71cbf4e29e3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:35.150161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:35.150990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:35.151369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:35.151801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:35.152601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:35.153002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:35.154480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:35.167856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 238/476] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:35.172293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e7159-507d-42bc-8cab-110db3f6a173 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:35.173568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:35.219521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:35.236120 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:35.237143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:35.237609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:35.238032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:35.238942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:35.240774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:35.250192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.sql.core [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 14:17:35.250192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 14:17:35.250192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [parameters: {'id': '40ea7af86c2d4b789cedb4062d16fca8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 14:17:35.250192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80150) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Aug 06 14:17:35.273885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return ref.to_dict() Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 14:17:35.275131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application yield resource Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise self.value Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application session.commit() Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 14:17:35.276271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.flush() Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application compat.raise_( Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 14:17:35.277369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 14:17:35.278456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 14:17:35.285753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [parameters: {'id': '40ea7af86c2d4b789cedb4062d16fca8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:35.288343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 14:17:35.293141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:35.295748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 239/477] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Aug 6 14:17:35 2022] POST /identity/v3/roles => generated 144 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:35.295748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:35.300500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a63b78-9b9f-48b6-bb12-73d407492564 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:35.340422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:35.341093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:35.341323 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:35.341594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:35.342279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:35.343777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:35.379509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 239/478] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:35 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:35.430434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea3f2-5137-4d30-9f7a-9091c579ad3a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:35.470699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4045b0-2617-4964-b001-7b0cd6b6a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:35.472017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4045b0-2617-4964-b001-7b0cd6b6a56f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:35.472504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4045b0-2617-4964-b001-7b0cd6b6a56f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:35.472976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4045b0-2617-4964-b001-7b0cd6b6a56f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:35.485519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4045b0-2617-4964-b001-7b0cd6b6a56f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-bd4045b0-2617-4964-b001-7b0cd6b6a56f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:35.491511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:35.492501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 14:17:35.493324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:35.496095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 240/479] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:17:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:35.502138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:35.517293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:35.618835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb405ab-1997-4230-9b51-e6a80e050376 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:35.656912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057e6c9-163c-4261-842d-ee536206d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:35.657897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e6c9-163c-4261-842d-ee536206d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:35.658378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e6c9-163c-4261-842d-ee536206d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:35.658854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e6c9-163c-4261-842d-ee536206d9a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:35.659789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057e6c9-163c-4261-842d-ee536206d9a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:35.660295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057e6c9-163c-4261-842d-ee536206d9a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:35.661867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057e6c9-163c-4261-842d-ee536206d9a3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:35.670744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 240/480] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:36.900836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388216e-95e1-41e5-9817-7ff9d53026b1 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:36.901813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388216e-95e1-41e5-9817-7ff9d53026b1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:36.902293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388216e-95e1-41e5-9817-7ff9d53026b1 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:36.903883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 241/481] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:36.909235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5b78-398c-4f50-b684-6c670383a56f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:36.910029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5b78-398c-4f50-b684-6c670383a56f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:36.910511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5b78-398c-4f50-b684-6c670383a56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:36.939813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0c3c5b78-398c-4f50-b684-6c670383a56f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:36.941335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0c3c5b78-398c-4f50-b684-6c670383a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:37.000859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e963f4d-bd89-4a40-9ac0-eef024ce05c9 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.001191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e963f4d-bd89-4a40-9ac0-eef024ce05c9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.001401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e963f4d-bd89-4a40-9ac0-eef024ce05c9 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.003015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 242/482] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:37.012374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06492695-ad0f-461b-b121-2ae12d0e2b20 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.012663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06492695-ad0f-461b-b121-2ae12d0e2b20 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.012938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06492695-ad0f-461b-b121-2ae12d0e2b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.031261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 241/483] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:37.040537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-06492695-ad0f-461b-b121-2ae12d0e2b20 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:37.041763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-06492695-ad0f-461b-b121-2ae12d0e2b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:37.096056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c5b78-398c-4f50-b684-6c670383a56f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:37.098411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:37.104030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:37.143547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 243/484] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:37.171597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06492695-ad0f-461b-b121-2ae12d0e2b20 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:37.273575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c5b78-398c-4f50-b684-6c670383a56f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:37.310537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:37.311209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.311490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.311762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.323490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:37.331192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:37.332158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:37.332922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 242/485] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:37.336479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:37.372965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06492695-ad0f-461b-b121-2ae12d0e2b20 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:37.444231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:37.445813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.446311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.446784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.452890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:37.467536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:37.472806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:37.473704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:37.474508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:37.482020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 244/486] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:37.487630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:37.527209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:37.541021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f97ac-7c8d-4930-9d94-70ce48ccd353 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:37.604069 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755f9b0-8b1b-4235-bf63-1879f509fc84 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:37.628585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:37.629136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.629422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.629700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.630422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:37.630730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:37.632124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:37.647482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:37.649634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.652471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.653005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.653946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:37.654449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:37.656432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:37.661139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 243/487] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:37.665265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:37.683140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 245/488] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:37.688472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:37.802324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104e1f5-c0c3-4f7e-8856-cda63411d795 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:37.827089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:37.836301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:37.839470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402121bc-a804-4422-b321-3f39b50625a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:37.840090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402121bc-a804-4422-b321-3f39b50625a9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.840368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402121bc-a804-4422-b321-3f39b50625a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.840702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402121bc-a804-4422-b321-3f39b50625a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.842619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271d6dc-2219-4cca-b223-4f2dbed02373 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:37.858578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402121bc-a804-4422-b321-3f39b50625a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:37.860061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402121bc-a804-4422-b321-3f39b50625a9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-402121bc-a804-4422-b321-3f39b50625a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:37.879055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:37.879908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:37.880741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:37.880741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:37.880741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:37.880741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:37.884856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 244/489] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:37.886080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4eca1-ba73-4718-af22-9da5951ed0b2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.886368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4eca1-ba73-4718-af22-9da5951ed0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.886640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4eca1-ba73-4718-af22-9da5951ed0b2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.888139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 245/490] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:37.888830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4eca1-ba73-4718-af22-9da5951ed0b2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:37.891058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c86fd4-e959-48de-912b-95f498e15dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:37.891997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c86fd4-e959-48de-912b-95f498e15dbb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.892539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c86fd4-e959-48de-912b-95f498e15dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.893084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c86fd4-e959-48de-912b-95f498e15dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.907626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c86fd4-e959-48de-912b-95f498e15dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:37.910734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c86fd4-e959-48de-912b-95f498e15dbb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-72c86fd4-e959-48de-912b-95f498e15dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:37.932860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:37.933937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:37.934966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:37.934966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:37.934966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:37.934966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:37.936810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 246/491] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:37.938224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115dfd-43be-423b-8a00-98ca0ece903a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:37.938635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115dfd-43be-423b-8a00-98ca0ece903a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:37.939080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115dfd-43be-423b-8a00-98ca0ece903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:37.960886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-70115dfd-43be-423b-8a00-98ca0ece903a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:37.962226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-70115dfd-43be-423b-8a00-98ca0ece903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:38.026203 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4eca1-ba73-4718-af22-9da5951ed0b2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.035391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 247/492] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:38.036553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115dfd-43be-423b-8a00-98ca0ece903a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.083504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.084344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.084726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.085105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.086512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:38.086814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.088266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.103577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 246/493] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:38.105094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.128737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.142935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.149786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115dfd-43be-423b-8a00-98ca0ece903a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.186386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443c4ed-1df1-4f88-831c-ce8d1b89948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.187041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c4ed-1df1-4f88-831c-ce8d1b89948f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.187325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c4ed-1df1-4f88-831c-ce8d1b89948f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.187601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443c4ed-1df1-4f88-831c-ce8d1b89948f None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.188292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443c4ed-1df1-4f88-831c-ce8d1b89948f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:38.188596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443c4ed-1df1-4f88-831c-ce8d1b89948f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.189972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443c4ed-1df1-4f88-831c-ce8d1b89948f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.206017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 248/494] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:38.206017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5d0e-4a6a-49c6-a611-ff75b8bb8bad None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.206017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5d0e-4a6a-49c6-a611-ff75b8bb8bad None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.206017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5d0e-4a6a-49c6-a611-ff75b8bb8bad None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.208223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 249/495] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:38.209038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5d0e-4a6a-49c6-a611-ff75b8bb8bad None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.221743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4df24e-331d-4fad-8a3a-51207d12a23a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.260593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.261233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.261534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.261801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.271866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:38.277208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:38.278055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:38.279278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 247/496] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:38.280254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.314278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5d0e-4a6a-49c6-a611-ff75b8bb8bad None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.347316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9d36e-eaec-4168-a0d7-9f54000a9d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.347956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d36e-eaec-4168-a0d7-9f54000a9d2a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.348237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d36e-eaec-4168-a0d7-9f54000a9d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.348514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d36e-eaec-4168-a0d7-9f54000a9d2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.355854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9d36e-eaec-4168-a0d7-9f54000a9d2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.357299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9d36e-eaec-4168-a0d7-9f54000a9d2a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.359757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b8a9d36e-eaec-4168-a0d7-9f54000a9d2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:38.370852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:38.371798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:38.372696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 250/497] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:38.373259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d36096-6803-4625-bffa-4bef89307758 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.373597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d36096-6803-4625-bffa-4bef89307758 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.373692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d36096-6803-4625-bffa-4bef89307758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.374366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.382279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2681-3429-4a2b-ac2a-fcad2ba131bc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.391274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-48d36096-6803-4625-bffa-4bef89307758 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:38.392471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-48d36096-6803-4625-bffa-4bef89307758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:38.418258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d61d93-7233-4536-b22f-606fa31ec24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.419032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61d93-7233-4536-b22f-606fa31ec24a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.419418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61d93-7233-4536-b22f-606fa31ec24a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.419696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61d93-7233-4536-b22f-606fa31ec24a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.429133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d61d93-7233-4536-b22f-606fa31ec24a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.430589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d61d93-7233-4536-b22f-606fa31ec24a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b1d61d93-7233-4536-b22f-606fa31ec24a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:38.443406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:38.444438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 14:17:38.445784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:38.447008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 248/498] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:38.448403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b07f-bf9e-47b4-91cb-bf4c08b73281 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.448685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b07f-bf9e-47b4-91cb-bf4c08b73281 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.448958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b07f-bf9e-47b4-91cb-bf4c08b73281 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.450214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 249/499] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:38.451009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899b07f-bf9e-47b4-91cb-bf4c08b73281 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.463488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 251/500] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:38.464610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d36096-6803-4625-bffa-4bef89307758 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.553028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899b07f-bf9e-47b4-91cb-bf4c08b73281 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.563427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d36096-6803-4625-bffa-4bef89307758 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.586293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.586937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.587221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.587500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.588203 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:38.588511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.589883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.591655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.597917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b202b10-3b9e-4c50-a263-1b4cc671d098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.598061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 250/501] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:38.598576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b202b10-3b9e-4c50-a263-1b4cc671d098 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.598921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b202b10-3b9e-4c50-a263-1b4cc671d098 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.599205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b202b10-3b9e-4c50-a263-1b4cc671d098 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.599827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.600007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b202b10-3b9e-4c50-a263-1b4cc671d098 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:38.600309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b202b10-3b9e-4c50-a263-1b4cc671d098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.601911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b202b10-3b9e-4c50-a263-1b4cc671d098 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.618168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.639119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 252/502] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:38.640360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb21fd-8d49-4365-9782-f634462c206b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.640649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb21fd-8d49-4365-9782-f634462c206b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.640927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb21fd-8d49-4365-9782-f634462c206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.659095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3ebb21fd-8d49-4365-9782-f634462c206b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:38.660300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3ebb21fd-8d49-4365-9782-f634462c206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:38.701289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84ed7b-efbe-4af0-91bc-7c14654e47a0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.733784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 253/503] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:38.733784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb21fd-8d49-4365-9782-f634462c206b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.743826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.744451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.744728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.745000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.745778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:38.746086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.747597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.784384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 251/504] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:38.785214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.831118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb21fd-8d49-4365-9782-f634462c206b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.848166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.864314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:38.864866 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.865520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.865799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.866073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.876567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:38.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:38.883210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 254/505] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:38.884171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.891166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95351de-c8a2-4df0-a8fa-f1cb5ef4d8da None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:38.923687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:38.924317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:38.924846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:38.925118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:38.935340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:38.936774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:38.951965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:38.952815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:38.953634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:38.953634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:38.953634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:38.953634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:38.953634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 252/506] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:38.954193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:38.980763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a97f4-4ea9-43d2-8175-9483ddb4d1ea None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.015603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.016213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.016495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.016766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.055575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549f61-2064-4072-83a2-d9a9b0b274ca None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.061674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.081755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.088230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.089925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.090734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.091841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.092325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.092813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.104865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.106856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:39.122674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:39.123634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:39.124472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:39.124472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:39.124472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.124472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:39.124472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 253/507] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:39.124739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.192071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 255/508] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:39.192900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.229669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fa44a-9af7-4b85-9913-3a9da53fd6e0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.262502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.263144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.263422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.263691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.267847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.289583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476635b6-1bd5-4888-b28f-06e224382fcc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.311515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.323646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.324258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.324534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.324806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.325504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:39.325805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.327209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.331536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.332960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.334852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 256/509] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:39.335666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.435261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f130672-51ea-4de5-8387-422e59e62134 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.435966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 254/510] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:39.436883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.468469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.469085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.469365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.469639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.470321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:39.470625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.472025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.480976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.484668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 257/511] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:39.485459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.521174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.541453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a478c46-0d3b-4432-9687-ad1aaf81e946 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.574393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.575039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.575320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.575594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.576277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:39.576580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.577935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.581650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dd442-37d1-4c6c-bf9d-0a6edcec8f36 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.589312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 255/512] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:39.590060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.615992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.616631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.616912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.617192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.627103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.628540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:39.643220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:39.644159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:39.645200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:39.645200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:39.645200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.645200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:39.645200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 258/513] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:39.645200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.691833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944b549-6bb3-4d74-8aa9-4df15598997e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.699076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.724526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.725155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.725434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.725707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.734766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.737223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.739206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.746418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba125747-98ef-46a1-aefb-290a7ad18523 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:39.753295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:39.754173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:39.759708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:39.759708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:39.759708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.759708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:39.759708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 256/514] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:39.759708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.779804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.780445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.780723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.780998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.791401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.792883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:39.807485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:39.808485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:39.809702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:39.809702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:39.809702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:39.809702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:39.809702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 259/515] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:39.809702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.862315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb8f9-5ffb-415b-936b-ce56cc6dbab2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.895139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.895780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.896062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.896336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.904247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.905692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.914493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bb804-5cb2-4b53-8ccc-240c420a3a05 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:39.916768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:39.917214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:39.918179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 257/516] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:39.919212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:39.950412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:39.951063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:39.951346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:39.951621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:39.952288 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:39.953768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:39.971193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:39.975031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 260/517] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:39 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:39.975824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.031476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b5898-0616-46b1-9b35-5f208e010954 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.067615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.068277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.068520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.068793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.069496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:40.069842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.071257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.083947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 258/518] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:40.084822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.098329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b4339-cec8-4ecd-9e72-23c2e7b047d1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.140807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.141442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.141726 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.142001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.142699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:40.143041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.144421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.157386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 261/519] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:40.158033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.198112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6a42b-4698-49cb-9ba4-f5644259b74a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.199457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:40.218093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:40.240460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.241084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.241359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.242200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.242928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:40.243218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.244569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.256540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 259/520] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:40.257615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.277294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce8c62-e793-415d-946b-502d7f5d74ce None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.313275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.313985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.314268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.314540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.315361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:40.315669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.317101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.336166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 262/521] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:40.337067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.370943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef332936-0d05-428a-980b-d9324cd56a0f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.406531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.407201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.407481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.407751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.415962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.418621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.426243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:40.435363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:40.436263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:40.437186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:40.437615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 260/522] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:40.438517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.461770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6da24-26f7-4f7a-bae5-49daef59676d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.475398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:40.500863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.501596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.501879 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.502173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.509185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.510686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:40.522316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:40.523759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 263/523] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:40.529207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.557275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19b0d-d375-4416-b57e-18e8c2a9024a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.594585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.595184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.595465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.595736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.603387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.604831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:40.616091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:40.617097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 261/524] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:40.622826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.640374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a2151-50d8-4c3f-8563-0b27bc9d2dce None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.677212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.677834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.678112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.678382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.688319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:40.722699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:40.732549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238eba1-7703-46ed-82af-5db54bbd55df None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.752543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.754036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.769341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.769965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.770245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.770519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.771286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:40.771597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.773555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.796249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 262/525] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:40.796249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.864543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 264/526] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:40.865367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.904430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:40.916398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579a6c1-fcb5-4492-89f7-6a6588309c26 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.955235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:40.955966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:40.956344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:40.956716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:40.957588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:40.958007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:40.959628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:40.966110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4605803-2877-4861-b822-30eb48285a26 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:40.982626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 263/527] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:40.982626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:40.990834 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:41.006981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.008209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.008582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.008975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.009820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:41.010222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.011766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.035259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 265/528] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:41.035259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:41.085041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42bc3-1df6-411a-8894-c79ad5847b93 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:41.120273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.120821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.121101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.121375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.122165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=075e9c0ef8fd406087d05a6ce36dc4e2) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:41.122477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.124001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.129985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 264/529] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:40 2022] GET /identity/v3/role_assignments?user.id=075e9c0ef8fd406087d05a6ce36dc4e2&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:41.130847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:41.162056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:41.167218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f80cfc-2376-42ff-b652-73c903cf4e3d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:41.206273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.206967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.207254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.207528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.220612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.222107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:41.240677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:41.242331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:41.243304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:41.243304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:41.243304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:41.243304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:41.247372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 266/530] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:41.248381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:41.250725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:41.292329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d21413-b3c8-4494-893c-6ebad00c6338 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:41.330749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.331400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.331681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.331957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.363978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c311b4b-80c0-4ab9-9fa8-bc49575628fc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:41.399475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.400326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.400754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.401171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.402096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=cf386f112e1b49fdba24871681961e0a) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:41.402579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.404241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.406961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.408527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.419183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 267/531] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/role_assignments?user.id=cf386f112e1b49fdba24871681961e0a&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:41.420439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:41.442250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:41.495899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:41.525092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 265/532] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:41.543512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:41.588031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c8f23-b76e-40a2-822d-d06d678d65f8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:41.626054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.626696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.627008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.627293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.628004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:41.628317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.629742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.650371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 268/533] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:41.656078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:41.719278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fecc6b-bdf2-4853-ac1b-18e19be497c9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:41.770360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:41.785443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.786398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.786893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.787520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.806575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.808555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.823587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a6142-e041-4c42-ac1c-7a625bd9022c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:41.840296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:41.841188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:41.842028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:41.842028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:41.842028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:41.842028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:41.846419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 266/534] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:41.853028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:41.890960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:41.892038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:41.892566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:41.893037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:41.906414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:41.907198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:41.908245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:41.931317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:41.932188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:41.933027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:41.934649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 269/535] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:41.942376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:42.040165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc5a3-f02c-42af-b20a-9977b0b3d86a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.100749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fc640-1fc3-4d49-b877-47bb2e4d54ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.101965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fc640-1fc3-4d49-b877-47bb2e4d54ce None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.102190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fc640-1fc3-4d49-b877-47bb2e4d54ce None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.102469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fc640-1fc3-4d49-b877-47bb2e4d54ce None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.105982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fc640-1fc3-4d49-b877-47bb2e4d54ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:42.106290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fc640-1fc3-4d49-b877-47bb2e4d54ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:42.108041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fc640-1fc3-4d49-b877-47bb2e4d54ce None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:42.119210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d30a8-e330-43a4-ac13-86acb23241ae None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.120723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 267/536] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:42.122603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774a32b-82e6-4ad8-917e-ad0443e88b64 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.122603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774a32b-82e6-4ad8-917e-ad0443e88b64 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.122771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774a32b-82e6-4ad8-917e-ad0443e88b64 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.124102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 268/537] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:42.128215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774a32b-82e6-4ad8-917e-ad0443e88b64 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:42.155922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb849f-9cb8-4695-8476-17f290c6952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.156590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb849f-9cb8-4695-8476-17f290c6952b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.156810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb849f-9cb8-4695-8476-17f290c6952b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.157080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb849f-9cb8-4695-8476-17f290c6952b None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.157817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb849f-9cb8-4695-8476-17f290c6952b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:42.158133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb849f-9cb8-4695-8476-17f290c6952b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:42.159589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb849f-9cb8-4695-8476-17f290c6952b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:42.161276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:42.178116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 270/538] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:42.178116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b540-42af-469f-b01a-6d9481a9d87a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.178116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b540-42af-469f-b01a-6d9481a9d87a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.178116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b540-42af-469f-b01a-6d9481a9d87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.199420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cc95b540-42af-469f-b01a-6d9481a9d87a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:42.200976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cc95b540-42af-469f-b01a-6d9481a9d87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:42.241647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:42.259350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774a32b-82e6-4ad8-917e-ad0443e88b64 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.296680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.297321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.297612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.297867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.301863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 271/539] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:42.306971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:42.306971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:42.308161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b540-42af-469f-b01a-6d9481a9d87a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:42.318472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:42.319376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:42.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 269/540] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:42.336548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:42.430690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b540-42af-469f-b01a-6d9481a9d87a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.448039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da137e8-65c7-4da4-a9e2-31a3952a54bf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.460869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:42.476375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.477332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.477638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.477985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.478861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=8722d97d3ddc49a082ef84060a30d1b8) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:42.479210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:42.480845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:42.485549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.486164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.486440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.486657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 272/541] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:42 2022] GET /identity/v3/role_assignments?user.id=8722d97d3ddc49a082ef84060a30d1b8&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:42.486757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.487468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:42.491734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:42.492054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:42.493669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:42.535109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 270/542] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:42.536119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:42.556042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:42.699982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb8e63-db5c-49e6-a1c5-8c804f0d4ac9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.709928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955e7d0-4924-4f72-be47-745b5c218cc1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.751330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.751946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.752224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.752497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.753278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=cd235744a7ee462eb13e383930de4199) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:42.753585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:42.755032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:42.756042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.756796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.757133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.757889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.766674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 271/543] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:42 2022] GET /identity/v3/role_assignments?user.id=cd235744a7ee462eb13e383930de4199&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:42.775818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:42.781479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:42.783679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:42.785597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 273/544] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:42.788927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:42.906242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:42.941664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaca907-1ae5-400f-b970-b9b9df0ad84c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:42.978252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:42.978891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:42.979175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:42.979446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:42.980152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:42.980451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:42.981796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:42.990708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 272/545] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:42.999978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:43.056374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:43.116120 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddf71a-8d70-4911-94b3-b423b93c6a8c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:43.160969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:43.161903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:43.162383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:43.162823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:43.174974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:43.176815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:43.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:43.193480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:43.194289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:43.194289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:43.194289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:43.194289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:43.196381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 274/546] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:43.204170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:43.326923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83449f9-badc-4d71-b505-21fbb4daa9b9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:43.360422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:43.366101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99441ec-d001-48cd-87e7-f2fa0bd97253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:43.367087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99441ec-d001-48cd-87e7-f2fa0bd97253 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:43.367625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99441ec-d001-48cd-87e7-f2fa0bd97253 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:43.368047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99441ec-d001-48cd-87e7-f2fa0bd97253 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:43.368879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99441ec-d001-48cd-87e7-f2fa0bd97253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:43.369369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99441ec-d001-48cd-87e7-f2fa0bd97253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:43.371048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99441ec-d001-48cd-87e7-f2fa0bd97253 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:43.383901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 273/547] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:43.392213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:43.411379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f718b-8cc6-414f-84d8-7c31dbd79f88 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:43.411878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f718b-8cc6-414f-84d8-7c31dbd79f88 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:43.412289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f718b-8cc6-414f-84d8-7c31dbd79f88 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:43.414338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 274/548] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:17:43.430595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae338beb-5ef1-41b8-bdab-a1d3b9c73600 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:43.431109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae338beb-5ef1-41b8-bdab-a1d3b9c73600 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:43.431559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae338beb-5ef1-41b8-bdab-a1d3b9c73600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:43.455708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ae338beb-5ef1-41b8-bdab-a1d3b9c73600 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:43.459304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ae338beb-5ef1-41b8-bdab-a1d3b9c73600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:43.536799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 275/549] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:43.567545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae338beb-5ef1-41b8-bdab-a1d3b9c73600 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:43.592047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:43.602208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011342b-bb41-41f3-b011-d526e11fe9a3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:43.662220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:43.663050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:43.663982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:43.664354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:43.672283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:43.673898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:43.684860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae338beb-5ef1-41b8-bdab-a1d3b9c73600 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:43.700935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:43.702600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:43.703907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 275/550] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:43.711937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:43.714459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:43.741026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:43.741984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:43.742427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:43.742903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:43.761090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:43.777382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:43.778465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:43.779393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:43.787535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 276/551] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:43.788777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:43.837610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a48aa-f405-4e87-aeb2-ecefa7549f3f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:43.885402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:43.886043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:43.886322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:43.886596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:43.887443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:43.887751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:43.889226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:43.919831 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 276/552] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Aug 6 14:17:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:43.925595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:43.953828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:43.974172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb6320-c142-4c7a-b90c-a386dd99dd77 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.015476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bbd85-f600-436c-8fbc-bfd322419953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.016463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bbd85-f600-436c-8fbc-bfd322419953 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.016945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bbd85-f600-436c-8fbc-bfd322419953 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.017440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bbd85-f600-436c-8fbc-bfd322419953 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.018340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bbd85-f600-436c-8fbc-bfd322419953 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:44.018854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bbd85-f600-436c-8fbc-bfd322419953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.020499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bbd85-f600-436c-8fbc-bfd322419953 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.029608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 277/553] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:44.031258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5931d-3b74-4265-bb4f-b027d557d5be None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.031749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5931d-3b74-4265-bb4f-b027d557d5be None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.032227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5931d-3b74-4265-bb4f-b027d557d5be None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.038017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 278/554] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:44.039077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5931d-3b74-4265-bb4f-b027d557d5be None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.062109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5bbca-0c3e-4694-9e0e-f018563da024 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.089902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:44.108703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.109284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.109561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.109835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.150297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5931d-3b74-4265-bb4f-b027d557d5be None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.189641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e795792-be9e-48be-889c-87672be37288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.190912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795792-be9e-48be-889c-87672be37288 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.191536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795792-be9e-48be-889c-87672be37288 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.192170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e795792-be9e-48be-889c-87672be37288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.205421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.206551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e795792-be9e-48be-889c-87672be37288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.207297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.208802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e795792-be9e-48be-889c-87672be37288 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0e795792-be9e-48be-889c-87672be37288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:44.225162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:44.226098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:44.226977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:44.226977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:44.226977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:44.226977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:44.229848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:44.230974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 279/555] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:44.232627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711169e8-a662-484f-be8c-674a8d172146 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.233169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711169e8-a662-484f-be8c-674a8d172146 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.233658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711169e8-a662-484f-be8c-674a8d172146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.253955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-711169e8-a662-484f-be8c-674a8d172146 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:44.255661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-711169e8-a662-484f-be8c-674a8d172146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:44.313746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 277/556] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 14:17:44.314632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.335046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 280/557] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:44.336237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711169e8-a662-484f-be8c-674a8d172146 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.381180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:44.452679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392d8af-8b3d-4e1f-b2ee-a63b0a7ed864 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.461485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711169e8-a662-484f-be8c-674a8d172146 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.478885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:44.506360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.507312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.507791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.507971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.508239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.508517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.508874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.509340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.510218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:44.510730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.512365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.520248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.522149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.524939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 278/558] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:44.525942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:44.543466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:44.544841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:44.545698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:44.545698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:44.545698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:44.545698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:44.548696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 281/559] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:44.549974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.642457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1d81-fef1-441d-a50c-7f8d128166a2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.653369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:44.680341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e69bb3-835d-4f95-89c6-5008a012b3e3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.681065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.681955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.682559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.683084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.694148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:44.700637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:44.701541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:44.702413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:44.710648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 279/560] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:44.711768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.716414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.717036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.717321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.717593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.726345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.727611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:44.739294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:44.740238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:44.741177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:44.745878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 282/561] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:44.746660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.761096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:44.826271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e3b91-c491-44a1-bc77-e2151e48ec10 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.853246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac458ed-9e6a-4a95-b91f-ce454f1a3805 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:44.869839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.870713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.871027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.871443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.872424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:44.872719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.874093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.889850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 280/562] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:44.889850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.894830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122eb19-fc80-43a4-b809-07b0cb3c8d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:44.895436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122eb19-fc80-43a4-b809-07b0cb3c8d8a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.895718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122eb19-fc80-43a4-b809-07b0cb3c8d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.895992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122eb19-fc80-43a4-b809-07b0cb3c8d8a None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.896715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122eb19-fc80-43a4-b809-07b0cb3c8d8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:44.897027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122eb19-fc80-43a4-b809-07b0cb3c8d8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:44.900673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122eb19-fc80-43a4-b809-07b0cb3c8d8a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:44.910983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:44.916945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 283/563] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:44.918478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abf4bb-c0b3-430d-9d1d-f50c39c26833 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:44.918820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abf4bb-c0b3-430d-9d1d-f50c39c26833 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:44.919204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abf4bb-c0b3-430d-9d1d-f50c39c26833 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:44.922988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 284/564] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:44.922988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abf4bb-c0b3-430d-9d1d-f50c39c26833 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:44.997770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.001755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56170-d850-443d-862a-1b7bd258c327 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.032969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abf4bb-c0b3-430d-9d1d-f50c39c26833 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.039896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.040518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.040795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.041063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.041846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:45.042147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.043602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.066674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 281/565] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:45.067573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.073109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4152eca-7ac2-4130-b8ca-9cbe2f38815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.073754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4152eca-7ac2-4130-b8ca-9cbe2f38815e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.074033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4152eca-7ac2-4130-b8ca-9cbe2f38815e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.074305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4152eca-7ac2-4130-b8ca-9cbe2f38815e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.081393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4152eca-7ac2-4130-b8ca-9cbe2f38815e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.082829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4152eca-7ac2-4130-b8ca-9cbe2f38815e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f4152eca-7ac2-4130-b8ca-9cbe2f38815e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:45.095911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:45.096813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:45.097692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 285/566] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:45.098903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533d678-f131-45cb-a788-47dbec777dac None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.098903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533d678-f131-45cb-a788-47dbec777dac None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.098903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533d678-f131-45cb-a788-47dbec777dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.117603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1533d678-f131-45cb-a788-47dbec777dac None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:45.118838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1533d678-f131-45cb-a788-47dbec777dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:45.176914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.178655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331840bf-3606-4f8a-9e93-b9487bb27a60 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.188580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 286/567] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:45.189211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533d678-f131-45cb-a788-47dbec777dac None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.215976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.216623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.216904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.217180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.228228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.229718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:45.244772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:45.245647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:45.246542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:45.246542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:45.246542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:45.246542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:45.246542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 282/568] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:45.246824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.258374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.299941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533d678-f131-45cb-a788-47dbec777dac None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.333416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63a624-bf7b-4077-9152-896bb06392f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.334046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63a624-bf7b-4077-9152-896bb06392f2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.334325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63a624-bf7b-4077-9152-896bb06392f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.334597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63a624-bf7b-4077-9152-896bb06392f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.344924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63a624-bf7b-4077-9152-896bb06392f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.345659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25e5a-3715-4e46-8672-1a02e465d5dc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.346361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63a624-bf7b-4077-9152-896bb06392f2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-cb63a624-bf7b-4077-9152-896bb06392f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:45.360853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:45.361767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:45.362642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:45.362642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:45.362642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:45.362642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:45.362642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 287/569] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:45.363505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8359-8e35-4904-be92-05c54c2712ca None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.363784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8359-8e35-4904-be92-05c54c2712ca None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.364055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c8359-8e35-4904-be92-05c54c2712ca None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.365278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 288/570] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:45.365962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c8359-8e35-4904-be92-05c54c2712ca None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.380965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.381587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.381867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.382664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.383438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:45.383744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.385184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.399692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 283/571] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:45.401223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.417735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.466368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.477150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c8359-8e35-4904-be92-05c54c2712ca None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.498166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304aeab-8fd2-4807-a063-1234c0c54d55 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.509941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.510579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.510862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.511174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.521481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:45.527322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:45.528349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:45.529156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 289/572] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:45.529851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.537349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1306825-85f9-4391-aa2b-fa7d37d3dae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.537968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306825-85f9-4391-aa2b-fa7d37d3dae3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.538248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306825-85f9-4391-aa2b-fa7d37d3dae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.538520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306825-85f9-4391-aa2b-fa7d37d3dae3 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.539241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306825-85f9-4391-aa2b-fa7d37d3dae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:45.539546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306825-85f9-4391-aa2b-fa7d37d3dae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.540914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306825-85f9-4391-aa2b-fa7d37d3dae3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.552258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 284/573] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:45.553529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985e1d8-5fde-4721-bed1-494fb1a0378d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.554298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985e1d8-5fde-4721-bed1-494fb1a0378d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.554581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985e1d8-5fde-4721-bed1-494fb1a0378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.571802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6985e1d8-5fde-4721-bed1-494fb1a0378d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:45.572975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6985e1d8-5fde-4721-bed1-494fb1a0378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:45.631219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72f9c2-757b-4905-8c97-36bc43bf0400 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.633211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.636960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 285/574] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:45.651062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985e1d8-5fde-4721-bed1-494fb1a0378d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.667475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.668102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.668379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.668655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.669344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:45.669648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.671032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.683192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 290/575] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:45.684023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.696493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.737023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985e1d8-5fde-4721-bed1-494fb1a0378d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.770178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.770788 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.771100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.771378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.772153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=add21760f9ed4b82b942c5dd9a6ed096) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:45.772463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.773931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.779139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 286/576] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/role_assignments?user.id=add21760f9ed4b82b942c5dd9a6ed096&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:45.779903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.781517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b46821-dd02-4c6c-a0c3-e91f2e94205c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.824282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.825254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.825606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.825955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.826801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:45.827207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.828694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.839125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 291/577] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:45.840126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:45.875557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.901586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9ab74-70c2-48df-b65c-3b6dc9cc47dd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.926587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:45.942066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:45.942717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:45.943032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:45.943309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:45.960227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:45.962445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:45.964302 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbde0f-e3f7-4847-9e81-73532e208dbe None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:45.976428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:45.977571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 14:17:45.979169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:45.982241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 287/578] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:45.983108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.001717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.002293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.002574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.002849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.013407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:46.019837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:46.020750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:46.021572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 292/579] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:46.022683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.091332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef19fdf-b476-4dc1-9163-5ca500fc4a0f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.109608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.126891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e6734-7c2a-4904-ae44-9604e77dee00 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.128614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.129743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.129743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.129969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.136521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.137964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:46.150420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:46.151455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:46.152576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:46.154312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 288/580] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:46.155251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.166181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.166945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.167285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.167637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.179637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.181192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:46.196078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:46.197151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:46.198044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:46.198044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:46.198044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:46.198044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:46.198044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 293/581] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:46.198320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.207888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.275631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecdff9-9f49-4ac2-b005-89efd4a1adb5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.309854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28739d2-3279-437b-99c2-7d5574a8ab16 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.310362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.310697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.311005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.311299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.312075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:46.312379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.313819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.334620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 289/582] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:46.334620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.349441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.350273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.350757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.351757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.352602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:46.353015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.354848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.357129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.369279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 294/583] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:46.369279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.436698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.439632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcde2d-ce41-4372-94b7-7869de5ecab7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.479426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57490af-7c21-4f02-a896-512098cb4202 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.479746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.480376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.480655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.480926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.481672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:46.482024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.483516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.502184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 290/584] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:46.502937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.520779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.522002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.522466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.523396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.524260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:46.524705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.526250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.538804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 295/585] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:46.539772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.604091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.610538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a646c5-6508-444a-a580-5f7c89700ddc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.642445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.643111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.643392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.643667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.648407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33141e2b-4950-4abc-aa9d-7c676720dbfb None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.654814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.656278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:46.670848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:46.671940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:46.673012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:46.673012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:46.673012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:46.673012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:46.676740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 291/586] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:46.677587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.687037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.687986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.688265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.688563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.691829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.700521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.701949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:46.716907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:46.717776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:46.718656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:46.718656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:46.718656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:46.718656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:46.718656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 296/587] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:46.719441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.783963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d142-aad9-413c-8e8d-d7497303d430 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.816933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.817603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.817878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.818146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/075e9c0ef8fd406087d05a6ce36dc4e2/roles/c5d40ea25aa5440d8170744359aaa18a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.822600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad6ede-b415-46fc-ae62-8ee0943e1148 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:46.833179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.833672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=075e9c0ef8fd406087d05a6ce36dc4e2, role_id=c5d40ea25aa5440d8170744359aaa18a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.836165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.856384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 292/588] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:46 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/075e9c0ef8fd406087d05a6ce36dc4e2/roles/c5d40ea25aa5440d8170744359aaa18a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:46.857221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.873556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:46.874954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:46.875481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:46.876000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:46.887063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:46.888971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:46.904711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:46.905830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:17:46.906789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:46.909773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 297/589] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:46.911135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:46.938097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:46.982751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383641e-c617-4383-97b8-d94e404f96df None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.020180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.020719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.021008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.021285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.021985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:47.022292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.024350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.037735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 293/590] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:47.037989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f115934-2eb5-41eb-82c8-7faa8c690e01 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.039489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.083615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.084242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.084519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.084799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.085524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:47.085945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.088093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.108263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 298/591] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:47.108263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.140364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:47.161668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44f2f4-0d85-40bf-ac50-bdb8a1e6e3d7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.193838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:47.211440 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.212716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.213547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.214042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.215067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:47.215589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.218147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.228400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f04713-c27d-4ed7-8d11-ec7c8cabf6e9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.243895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 294/592] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:47.245321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.265346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.265990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.266270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.266537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.273405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.274814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:47.286117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:47.287039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:47.287963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 299/593] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:47.288497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.383570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:47.388515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2ff6e-2a4f-4be9-9c7c-1dc1a7b47327 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.394179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5fe15-af4f-4914-86f0-ca7fc67584b2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.437458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.438434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.438807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.439233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.439348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.440141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.440567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.440987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.447333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.448935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.454827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.456728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:47.461658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:47.463018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:47.464352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 295/594] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:47.465903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:47.480217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:47.481367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:47.482479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:47.482479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:47.482479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:47.482479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:47.484321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 300/595] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:47.485326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.529432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:47.604759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbb58c-864c-41ba-88c1-75432cfdc46d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.610155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e086da-2f48-4ac2-843f-dc8f7c32656b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.645844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.646680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.647001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.647369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.648404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:47.648811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.650456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.655559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.656388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.656848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.657275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.658182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:47.658635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.660791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.669027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:47.673429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 301/596] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:47.674898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.684773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 296/597] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:47.684773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.783497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9014c-92a5-4569-aae8-addf3b6c0100 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.793184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:47.796801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2585643-30dd-4ce8-ba2e-a72cf28a87a1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.821046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.821662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.821940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.822211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.822913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:47.823223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.824561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.833655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb08c6b-b6e5-4c96-b032-48923b3641bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.834491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb08c6b-b6e5-4c96-b032-48923b3641bb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.835496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb08c6b-b6e5-4c96-b032-48923b3641bb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.835944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb08c6b-b6e5-4c96-b032-48923b3641bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.836340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 302/598] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:47.837288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.856794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb08c6b-b6e5-4c96-b032-48923b3641bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:47.858182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb08c6b-b6e5-4c96-b032-48923b3641bb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5bb08c6b-b6e5-4c96-b032-48923b3641bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:47.872599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:47.873500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:47.874408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:47.874408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:47.874408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:47.874408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:47.874408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 297/599] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:47.875118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885a79e-3825-4e46-a62d-bd97cecf901a None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.875220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885a79e-3825-4e46-a62d-bd97cecf901a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.875432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885a79e-3825-4e46-a62d-bd97cecf901a None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:47.881130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 298/600] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:47.881130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885a79e-3825-4e46-a62d-bd97cecf901a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:47.906987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:47.954091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ae2e-072e-4fec-9286-88848a42def0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.986828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885a79e-3825-4e46-a62d-bd97cecf901a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:47.991312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:47.991989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:47.992270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:47.992544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/cf386f112e1b49fdba24871681961e0a/roles/c5d40ea25aa5440d8170744359aaa18a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.010801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=cf386f112e1b49fdba24871681961e0a, role_id=c5d40ea25aa5440d8170744359aaa18a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.012458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.027501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77bff-8632-4853-9e2e-e5250241d8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.039120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 303/601] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:47 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/cf386f112e1b49fdba24871681961e0a/roles/c5d40ea25aa5440d8170744359aaa18a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:48.039358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77bff-8632-4853-9e2e-e5250241d8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.039358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77bff-8632-4853-9e2e-e5250241d8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.039358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77bff-8632-4853-9e2e-e5250241d8d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.039590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.047395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77bff-8632-4853-9e2e-e5250241d8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.048999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77bff-8632-4853-9e2e-e5250241d8d5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-49d77bff-8632-4853-9e2e-e5250241d8d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:48.062066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:48.063063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:48.063966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:48.067048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 299/602] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:48.069843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3718e1-ee07-4249-bd34-a1755d52580b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.070128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3718e1-ee07-4249-bd34-a1755d52580b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.070405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3718e1-ee07-4249-bd34-a1755d52580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.092177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-db3718e1-ee07-4249-bd34-a1755d52580b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:48.093923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-db3718e1-ee07-4249-bd34-a1755d52580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:48.105422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:48.169340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72d610-36c9-40bf-bc8c-6b06390216d1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.177554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 300/603] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:48.178261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3718e1-ee07-4249-bd34-a1755d52580b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.182369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:48.212219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.213153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.213736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.214198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.215191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:48.215762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.217293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.230547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 304/604] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:48.234901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.294651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3718e1-ee07-4249-bd34-a1755d52580b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.339067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.339730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.340013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.340289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.341586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:48.342041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.343686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.350373 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ad9ea-bd1a-44a9-9a43-5b889a7823e1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.362722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 301/605] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:48.362722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.367314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:48.396835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.398155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.398442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.398729 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.411714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:48.420282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:48.421249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:48.422067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:48.426657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 305/606] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:48.427672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.469069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:48.493763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52dc6-7e6a-466e-989a-f3ca93ccdbc8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.534208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.535312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.535935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.536412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.544757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.546696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.563530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63d09-fef4-43c5-bfca-b32a804940a5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:48.563910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:48.564758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:48.565721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:48.566551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 302/607] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:48.574854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.601571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.602255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.602540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.602821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/8722d97d3ddc49a082ef84060a30d1b8/roles/c5d40ea25aa5440d8170744359aaa18a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.618877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=8722d97d3ddc49a082ef84060a30d1b8, role_id=c5d40ea25aa5440d8170744359aaa18a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.620386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.644666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 306/608] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:48 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/8722d97d3ddc49a082ef84060a30d1b8/roles/c5d40ea25aa5440d8170744359aaa18a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:48.645666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.666082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:48.747535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6a60f-b5aa-49f0-b4a0-a60b8a6f667d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.747881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:48.768786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9cf6-2fc2-45d0-af68-20f23269f3dc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.807752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.808381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.808759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.809041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.809800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:48.810109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.811807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.813780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.814414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.814791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.815115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:48.815830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:48.816133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:48.817498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:48.830143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 307/609] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:48.831552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.834348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 303/610] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:48.858904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:48.951632 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80110a-49a7-4406-8c90-ddbae113742a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:48.997624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d27c8-ebaf-4bb5-8ba2-2a9847f89849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:48.998227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d27c8-ebaf-4bb5-8ba2-2a9847f89849 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:48.998533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d27c8-ebaf-4bb5-8ba2-2a9847f89849 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:48.998812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d27c8-ebaf-4bb5-8ba2-2a9847f89849 None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/cd235744a7ee462eb13e383930de4199/roles/c5d40ea25aa5440d8170744359aaa18a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.011897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:49.018024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d27c8-ebaf-4bb5-8ba2-2a9847f89849 None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=cd235744a7ee462eb13e383930de4199, role_id=c5d40ea25aa5440d8170744359aaa18a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:49.020188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d27c8-ebaf-4bb5-8ba2-2a9847f89849 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:49.029116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82e8f9-f0a2-4576-9bc2-cb63dc64eb02 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:49.042954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 308/611] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:48 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/cd235744a7ee462eb13e383930de4199/roles/c5d40ea25aa5440d8170744359aaa18a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:17:49.044736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa072-ea22-4c07-bb90-161a96717f48 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.045231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa072-ea22-4c07-bb90-161a96717f48 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.045778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa072-ea22-4c07-bb90-161a96717f48 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.056897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 309/612] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:49 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:49.059020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a421523-e4f9-471e-8680-a83ae8e57b0e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.059530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a421523-e4f9-471e-8680-a83ae8e57b0e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.060967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a421523-e4f9-471e-8680-a83ae8e57b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.083341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:49.099318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1a421523-e4f9-471e-8680-a83ae8e57b0e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:49.104261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:49.106103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1a421523-e4f9-471e-8680-a83ae8e57b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:49.107127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.107624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.108111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.132241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:49.136672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:49.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:49.164447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:49.165318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:49.165318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:49.165318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:49.165318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:49.167674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 304/613] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:49.172994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:49.251711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 310/614] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:49.284871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a421523-e4f9-471e-8680-a83ae8e57b0e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:49.334822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271f6b0-0189-4340-837d-9b53ff93aab8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:49.373328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:49.374297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.374792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.375286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.376214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:49.376711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:49.378375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:49.392406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 305/615] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:49.413892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:49.426742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a421523-e4f9-471e-8680-a83ae8e57b0e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:49.450474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:49.468759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:49.469347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:49.470219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.470719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.471199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.482272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:49.487430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:49.488328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:49.489127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:49.492221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 311/616] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:49.497990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:49.556593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff58ea6-4dc6-41b0-b643-c52a5b8ba4fe None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:49.608963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:49.609973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.610447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.610977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.616047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f415a-2d2a-43c3-8304-69f57e693fae None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:49.619608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:49.621278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:49.635183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:49.636139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:17:49.637076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:49.639654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 306/617] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:49.645923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:49.670214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:49.671218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.671796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.672305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.673275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:49.673704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:49.675709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:49.690061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 312/618] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:49.697797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:49.783259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:49.853391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813395a0-410e-49e1-9268-d4d35f54eca9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:49.887239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:49.893931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:49.894712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.895171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.896757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:49.899474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:49.900620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:49.904128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:49.917447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412da72-720a-4e02-bbd7-d97f17b7d83b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:49.938383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 307/619] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Aug 6 14:17:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:49.943619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:49.985216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:49.985873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:49.986155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:49.986430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.014057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.015539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:50.044580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:50.045858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:50.046964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:50.046964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:50.046964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:50.046964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:50.050824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 313/620] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:50.052457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.140991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0d456-fc31-49a3-9e43-817d3af63c23 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.208573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.209228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.209512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.209790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.229309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.230764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.233329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:50.247447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792fb0b-bee5-42b1-b581-deaa081a0cdc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:50.264174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:50.265072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:50.265927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:50.265927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:50.265927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:50.265927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:50.265927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 308/621] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:50.271446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.287232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:50.314043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.314667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.314986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.315257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.315946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:50.316247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.317602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.343555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 314/622] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:50.350836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.421718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d02ccd-1361-496f-b81b-ee648cb36d34 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.457805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b003d-6e72-4905-8797-705026366041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.458420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b003d-6e72-4905-8797-705026366041 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.458697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b003d-6e72-4905-8797-705026366041 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.459100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b003d-6e72-4905-8797-705026366041 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.459916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b003d-6e72-4905-8797-705026366041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:50.460219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b003d-6e72-4905-8797-705026366041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.461785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b003d-6e72-4905-8797-705026366041 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.465857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76385-ea06-4817-a55c-0905920d5c8f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.476078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 309/623] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:50.477554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845454-5959-408a-916a-e0f3bd5fdab4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.477962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845454-5959-408a-916a-e0f3bd5fdab4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.478269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845454-5959-408a-916a-e0f3bd5fdab4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.485735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 310/624] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:50.486905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845454-5959-408a-916a-e0f3bd5fdab4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.505372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c54b9d-3b4a-44c4-846f-dcbf994587ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.506019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c54b9d-3b4a-44c4-846f-dcbf994587ab None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.506298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c54b9d-3b4a-44c4-846f-dcbf994587ab None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.506572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c54b9d-3b4a-44c4-846f-dcbf994587ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.516905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c54b9d-3b4a-44c4-846f-dcbf994587ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.518335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c54b9d-3b4a-44c4-846f-dcbf994587ab None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-09c54b9d-3b4a-44c4-846f-dcbf994587ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:50.530571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:50.531533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:50.532460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 315/625] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:50.533519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfdd52-4c4b-4976-a69a-e11de3f594f9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.533519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfdd52-4c4b-4976-a69a-e11de3f594f9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.533805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfdd52-4c4b-4976-a69a-e11de3f594f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.552897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-33cfdd52-4c4b-4976-a69a-e11de3f594f9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:50.554636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-33cfdd52-4c4b-4976-a69a-e11de3f594f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:50.564472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:50.569886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:50.619285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845454-5959-408a-916a-e0f3bd5fdab4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.632829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 316/626] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:50.632829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfdd52-4c4b-4976-a69a-e11de3f594f9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.680779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.681656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.682097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.682524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.689936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.691634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:50.704051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:50.705198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:50.706199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:50.708269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 311/627] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:50.709184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.732469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfdd52-4c4b-4976-a69a-e11de3f594f9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.768480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.768988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.769274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.769547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.770332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:50.770636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.772096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.791441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 317/628] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:50.792290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.794996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:50.816531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06c16-2ba0-4ed5-8bc3-e5870bd4c102 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.844019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:50.855327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.855976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.856254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.856525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.870025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:50.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:50.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:50.878180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:50.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 312/629] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:50.883561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:50.924939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c26caf-ba26-448c-9b2e-b9d557670402 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:50.961176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:50.962030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:50.962472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:50.962923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:50.963874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:50.964322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:50.965992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:50.979624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 318/630] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:50.980674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.004729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa250174-7970-4b32-bcc8-97c834bab62b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.041729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36be33-9093-45cd-81da-a6022e907c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.042512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36be33-9093-45cd-81da-a6022e907c1f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.042794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36be33-9093-45cd-81da-a6022e907c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.043109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36be33-9093-45cd-81da-a6022e907c1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.045944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.088938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c3da6-674a-42fd-b435-08b101a23fd2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.112230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.125292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.126183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.126291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36be33-9093-45cd-81da-a6022e907c1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.126509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.126739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.127475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:51.127773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.127869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36be33-9093-45cd-81da-a6022e907c1f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.129120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.139178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 319/631] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:51.140341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.244042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 313/632] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:51.245525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8766d5-4831-43ca-b08e-e147124baf86 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.245828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8766d5-4831-43ca-b08e-e147124baf86 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.246112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8766d5-4831-43ca-b08e-e147124baf86 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.248825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33634-97ec-48db-a07b-599ee1ef3527 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.251616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 314/633] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:51.252390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8766d5-4831-43ca-b08e-e147124baf86 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.275765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.287285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.287961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.288241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.288514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/add21760f9ed4b82b942c5dd9a6ed096/roles/c5d40ea25aa5440d8170744359aaa18a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.302700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=add21760f9ed4b82b942c5dd9a6ed096, role_id=c5d40ea25aa5440d8170744359aaa18a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.304172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.316848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 320/634] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:17:51 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/add21760f9ed4b82b942c5dd9a6ed096/roles/c5d40ea25aa5440d8170744359aaa18a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:17:51.317633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.342348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.360489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8766d5-4831-43ca-b08e-e147124baf86 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.399140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b2089-ca53-4320-bd5d-1f4ce0af72aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.399829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b2089-ca53-4320-bd5d-1f4ce0af72aa None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.400112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b2089-ca53-4320-bd5d-1f4ce0af72aa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.400380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b2089-ca53-4320-bd5d-1f4ce0af72aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.411994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b2089-ca53-4320-bd5d-1f4ce0af72aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.413535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b2089-ca53-4320-bd5d-1f4ce0af72aa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-629b2089-ca53-4320-bd5d-1f4ce0af72aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:51.429438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:51.430392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:51.431277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:51.431277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:51.431277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:51.431277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:51.431277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 315/635] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:51.432082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038735fa-7023-4490-bbc5-c7f3a04e7639 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.432183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038735fa-7023-4490-bbc5-c7f3a04e7639 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.432434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038735fa-7023-4490-bbc5-c7f3a04e7639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.442630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35b2d5-df45-4afc-bb85-13305092337c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.452096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-038735fa-7023-4490-bbc5-c7f3a04e7639 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:51.453340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-038735fa-7023-4490-bbc5-c7f3a04e7639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:51.485687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a1a4a-7605-4070-94b0-106640209e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.486328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a1a4a-7605-4070-94b0-106640209e0d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.486601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a1a4a-7605-4070-94b0-106640209e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.486953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a1a4a-7605-4070-94b0-106640209e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.499434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a1a4a-7605-4070-94b0-106640209e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.500881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a1a4a-7605-4070-94b0-106640209e0d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-604a1a4a-7605-4070-94b0-106640209e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:51.516626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:51.517636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:51.518532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:51.518532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:51.518532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:51.518532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:51.518532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 321/636] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:51.519257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cd288-42ce-482e-8d29-363ce2f1a361 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.519257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cd288-42ce-482e-8d29-363ce2f1a361 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.519519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cd288-42ce-482e-8d29-363ce2f1a361 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.520866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 322/637] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:51.521397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cd288-42ce-482e-8d29-363ce2f1a361 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.538847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 316/638] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:51.539707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038735fa-7023-4490-bbc5-c7f3a04e7639 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.561009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.619229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.643560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cd288-42ce-482e-8d29-363ce2f1a361 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.656779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038735fa-7023-4490-bbc5-c7f3a04e7639 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.681660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b806ed-be32-4df7-9f97-76742bc7408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.682410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b806ed-be32-4df7-9f97-76742bc7408b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.682635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b806ed-be32-4df7-9f97-76742bc7408b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.682949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b806ed-be32-4df7-9f97-76742bc7408b None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.683671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b806ed-be32-4df7-9f97-76742bc7408b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:51.683976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b806ed-be32-4df7-9f97-76742bc7408b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.685330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b806ed-be32-4df7-9f97-76742bc7408b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.692729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.693349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.693626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.693895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.695694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:51.695935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.697242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.702426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 323/639] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:51.702426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19b63b-e7ab-4671-8c32-ad6c08bd46a6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.702426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19b63b-e7ab-4671-8c32-ad6c08bd46a6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.702426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19b63b-e7ab-4671-8c32-ad6c08bd46a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.717094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 317/640] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:51.717094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.721875 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-aa19b63b-e7ab-4671-8c32-ad6c08bd46a6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:51.723126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aa19b63b-e7ab-4671-8c32-ad6c08bd46a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:51.795394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 324/641] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:51.795935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19b63b-e7ab-4671-8c32-ad6c08bd46a6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.814208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.834202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af4cc9-4e4f-44a5-8c3e-dae17e1d5bd7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.862009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:51.872754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.873391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.873982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.873982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.888592 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:51.894613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:51.895641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:51.896585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:51.902832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 318/642] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:51.902832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:51.908290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19b63b-e7ab-4671-8c32-ad6c08bd46a6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:51.947184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:51.947857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:51.948130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:51.948407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:51.956418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:51.957857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:51.970379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:51.976723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 325/643] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:51.977891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.016367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a5575-8ae5-45c0-b97e-392fd26dad1b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.053036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.053679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.054450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.054750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.058640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.061567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.063067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:52.074389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:52.075449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:52.076643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:52.080334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 319/644] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:52.081173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.087994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3329e-7da0-478e-a57e-042f7f509d77 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.126952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.127815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.128456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.128739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.130472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.142116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:52.147420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:52.148358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:52.149191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 326/645] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:52.151475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.190283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78cf1-047f-471f-8a84-a83f40034ce4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.225190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.225834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.226115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.226390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.227131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:52.227436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.228803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.237073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 320/646] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:52.238152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.260329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a83a9-81ee-4beb-8ef6-c4616041ed66 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.297356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.297972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.298250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.298521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.299398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:52.299965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.301392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.304663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.320938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 327/647] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:52.322209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.349494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28dd51-897b-4744-a1de-ac33d1d5d678 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.361342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.390572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.391307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.391802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.392078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.392812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:52.393118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.394538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.412817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 321/648] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:52.413728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.430412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0657-c36b-450b-b6ba-2f4402df0333 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.467484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.468115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.468392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.468660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.469371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:52.469675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.471055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.486584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 328/649] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:52.486584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.524942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83701b7b-4a10-4c59-bcd6-3adeb708d205 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.542013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.562856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.563567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.563916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.564290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.575003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.576449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.587150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd88a6-edca-42a4-9f30-d8a14d55add3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:52.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:52.592218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:52.597960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:52.597960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:52.597960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:52.597960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:52.597960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 322/650] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:52.597960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.621193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.628039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.628787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.629167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.629554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.699505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.700942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.701859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92c8a-719b-40a7-bfc5-f210393411a8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.735937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.736556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.736845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.737126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.737906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=075e9c0ef8fd406087d05a6ce36dc4e2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:52.738220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.739720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.745996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 323/651] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/role_assignments?user.id=075e9c0ef8fd406087d05a6ce36dc4e2&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:52.746773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.770342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.834070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 329/652] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:52.835051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:52.869643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:52.889834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207dc66-831e-452f-b811-f7235891e5ac None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.941275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:52.942347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:52.942640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:52.942951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:52.956252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:52.957821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:52.971718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52d5c4-f415-4024-9258-23da6f07f68a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:52.974783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:52.976433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:52.977544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:52.977544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:52.977544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:52.977544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:52.978851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 324/653] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:52.979921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.015900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.017468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.018505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.019128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.021049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:53.021354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.023295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.039666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 330/654] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:53.040882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.068061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.108883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6b606-9b70-488d-a320-818c710a4495 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.144805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.145626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.146095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.146474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.158631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.160435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.162385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d133-bd47-40d8-9fda-7fb3d649dc38 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:53.176842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:53.177871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:53.178998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:53.178998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:53.178998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:53.178998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:53.180850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 325/655] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:53.181746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.195305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.201362 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.202203 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.202624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.203196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.204053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:53.204561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.206079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.223711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 331/656] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:53.224890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.290856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f544b6-19c3-4df4-a9d1-855081761af6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.303501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.328205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.328558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cf7ba-e650-4c95-9ea1-b1777b6bf1a5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.329015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.329362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.329705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.338638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.340377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:53.353380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:53.354365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:53.355316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 326/657] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:53.355902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.371597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.372888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.373345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.373801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.374667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:53.375162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.376687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.389986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 332/658] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:53.391713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.411817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.468461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6eae52-8b26-4968-8720-de894d20c88e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.499731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0c4bc-9d73-430d-8ebb-8b16edb14eaa None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.505127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b52420-66a9-405d-ac9f-0f2463f2eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.505767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52420-66a9-405d-ac9f-0f2463f2eaec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.506048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52420-66a9-405d-ac9f-0f2463f2eaec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.506318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52420-66a9-405d-ac9f-0f2463f2eaec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.515237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b52420-66a9-405d-ac9f-0f2463f2eaec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.516668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b52420-66a9-405d-ac9f-0f2463f2eaec None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-c5b52420-66a9-405d-ac9f-0f2463f2eaec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:53.528446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:53.529412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:53.530334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 327/659] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:53.530958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c0106-e0d6-4dda-8c75-faa1627e1eeb None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.531025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c0106-e0d6-4dda-8c75-faa1627e1eeb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.531875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c0106-e0d6-4dda-8c75-faa1627e1eeb None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.533327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 328/660] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:53.537801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c0106-e0d6-4dda-8c75-faa1627e1eeb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.541969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.542587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.542893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.543178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.543961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:53.544260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.545651 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.549554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.571183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 333/661] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:53.572042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.649175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c0106-e0d6-4dda-8c75-faa1627e1eeb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.659633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.681603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57210b7-7a37-4089-a48b-70b70fb08545 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.684313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9effe-7ad4-400b-b9fd-1a4b4c7931f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.684949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9effe-7ad4-400b-b9fd-1a4b4c7931f1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.685223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9effe-7ad4-400b-b9fd-1a4b4c7931f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.685495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9effe-7ad4-400b-b9fd-1a4b4c7931f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.692012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9effe-7ad4-400b-b9fd-1a4b4c7931f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.693447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9effe-7ad4-400b-b9fd-1a4b4c7931f1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-66e9effe-7ad4-400b-b9fd-1a4b4c7931f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:53.704945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:53.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:53.706756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 329/662] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:53.707298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b369ef-87a9-49f4-81e5-c077e65b10bd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.707431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b369ef-87a9-49f4-81e5-c077e65b10bd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.707703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b369ef-87a9-49f4-81e5-c077e65b10bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.725963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-77b369ef-87a9-49f4-81e5-c077e65b10bd None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:53.727162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-77b369ef-87a9-49f4-81e5-c077e65b10bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:53.727368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.728142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.728565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.728992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.730255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:53.730564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.732006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.751465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 334/663] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:53.752264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.798972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 330/664] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:53.800237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b369ef-87a9-49f4-81e5-c077e65b10bd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.810214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.868910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4be1-8253-4170-8164-aa23a8fb3b1c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.906078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.906620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.906925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.907202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.914816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:53.918528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b369ef-87a9-49f4-81e5-c077e65b10bd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:53.955829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:53.956798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:53.957180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:53.957423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:53.958177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:53.958485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.959975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:53.971975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 331/665] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:53.972787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:53.983695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:53.985464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.042519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.079112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63413716-3130-459b-b36e-c94d95c4e01a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.098962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 335/666] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:54.098962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.118739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.119407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.119685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.119946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.148805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.192550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:54.193924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.204558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d31cb-4110-48a3-b4a8-5a03d2aaa497 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.240614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.241859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.242138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.242414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.253065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:54.258148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:54.259103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:54.259918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 336/667] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:54.260591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.278130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.299564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 332/668] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:54.300563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.359892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.376864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e372-98a1-422c-a897-ae430377e662 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.404259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baea73e-4ecb-45cb-9116-ce7385e4ef52 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.414837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.416092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.416554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.417001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.418296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=cf386f112e1b49fdba24871681961e0a) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:54.418785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:54.420570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.427653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 337/669] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/role_assignments?user.id=cf386f112e1b49fdba24871681961e0a&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:54.429329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.455952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.457021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.457513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.458031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.470292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:54.471985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:54.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:54.490603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:54.492515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:54.492515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:54.492515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:54.492515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:54.495384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 333/670] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:54.496462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.551052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.580169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243c59-b148-48a5-827a-246ca9f4e3a1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.631522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.632520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.634036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.634036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.634036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:54.634263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:54.635564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.642784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345c8c4-9f44-4747-bd1f-12f878b0eefd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.646997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 338/671] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:54.646997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.661691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.690595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.691616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.692110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.692456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.705847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:54.707408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:54.724333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:54.725247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:54.726774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:54.726774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:54.726774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:54.726774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:54.726774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 334/672] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:54.727085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.784285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b302b-31aa-49d3-b4a0-d546e48abc71 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.821371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.822447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.822952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.823578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.824539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:54.825036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:54.826735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.834583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.840847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904d52-d063-4fab-b905-0f5636265783 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:54.854930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 339/673] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:54.854930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.892359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:54.893253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:54.893929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:54.894215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:54.908243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:54.910418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:54.930951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:54.931911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:54.932804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:54.932804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:54.932804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:54.932804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:54.933463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 335/674] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:54.934408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:54.958389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:54.979321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180ad75-ea65-4f70-a23c-dbb58fb07315 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.013786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.014401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.014683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.014987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.015672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:55.015973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.017354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.029378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 340/675] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:55.030176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.053447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde547f4-1277-46fb-b883-9fe98fad72c2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.092264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f0a2c-b281-42c8-904c-08213c4208e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.092910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0a2c-b281-42c8-904c-08213c4208e7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.093193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0a2c-b281-42c8-904c-08213c4208e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.093463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0a2c-b281-42c8-904c-08213c4208e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.095583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:55.101328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f0a2c-b281-42c8-904c-08213c4208e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.103369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f0a2c-b281-42c8-904c-08213c4208e7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e98f0a2c-b281-42c8-904c-08213c4208e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:55.116478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:55.117551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:55.118614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 336/676] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:55.119139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf9166-f4ba-481b-a4b0-30fd6c7186d6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.119406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf9166-f4ba-481b-a4b0-30fd6c7186d6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.119678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf9166-f4ba-481b-a4b0-30fd6c7186d6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.121526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 337/677] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:55.122237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf9166-f4ba-481b-a4b0-30fd6c7186d6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.149613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d78868-4f83-42cc-979e-ecc0f64594f4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.184624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.185247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.185530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.185804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.186598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:55.187052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.188408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.205634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 341/678] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:55.205634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.208174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:55.237694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf9166-f4ba-481b-a4b0-30fd6c7186d6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.274162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f484848-dc69-4d29-9097-6cd219f36d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.274803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f484848-dc69-4d29-9097-6cd219f36d4b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.275128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f484848-dc69-4d29-9097-6cd219f36d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.275402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f484848-dc69-4d29-9097-6cd219f36d4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.282240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f484848-dc69-4d29-9097-6cd219f36d4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.283729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f484848-dc69-4d29-9097-6cd219f36d4b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9f484848-dc69-4d29-9097-6cd219f36d4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:55.295671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:55.296521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:55.297436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 338/679] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:55.298558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8dbf2-a887-42bb-a12b-9fc1a7766d74 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.298558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8dbf2-a887-42bb-a12b-9fc1a7766d74 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.298840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8dbf2-a887-42bb-a12b-9fc1a7766d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.317215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8fb8dbf2-a887-42bb-a12b-9fc1a7766d74 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:55.318411 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8fb8dbf2-a887-42bb-a12b-9fc1a7766d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:55.320138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e78a0f-b663-4871-8742-db13be25f022 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.338504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:55.357272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.357885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.358167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.358443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.359217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:55.359527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.361073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.379142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 342/680] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:55.380010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.398910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 339/681] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:55.398910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8dbf2-a887-42bb-a12b-9fc1a7766d74 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.473249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:55.489316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538715fd-8d2e-460e-9901-b8803f36e002 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.511326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8dbf2-a887-42bb-a12b-9fc1a7766d74 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.524150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.524891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.525257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.525534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.533086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.534517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:55.545803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:55.546979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:55.547945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 343/682] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:55.548551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.550476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.551122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.551395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.551664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.552405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:55.552706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.554621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.566843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 340/683] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:55.571771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.593936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:55.657159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691ed9a-4414-4204-8e71-7e7bac95b7db None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.684020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368275f6-5d9e-4eec-bc8d-da3d67b64abc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.692778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.693392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.693780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.694052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.694888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=8722d97d3ddc49a082ef84060a30d1b8) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:55.695201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.696609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.702486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 344/684] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/role_assignments?user.id=8722d97d3ddc49a082ef84060a30d1b8&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:55.703464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.724877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:55.741043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.742324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.742620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.742933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.759370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:17:55.766398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:55.767465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:17:55.768293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:55.771056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 341/685] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:55.772273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.839482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5840f0f-c2d9-4ddc-86cc-9a1569d0643e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.841585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:55.888978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.889940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.890418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.891520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.892521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:55.893032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.894627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.900928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7237-af1d-4a64-a780-94abebb5097a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:55.915320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 345/686] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:55.916421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:55.952284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:55.952913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:55.953190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:55.953459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:55.954240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=cd235744a7ee462eb13e383930de4199) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:55.954540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:55.956254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:55.962914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 342/687] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/role_assignments?user.id=cd235744a7ee462eb13e383930de4199&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:55.963857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:56.023488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:56.046718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c03a0-e608-4882-8f5d-a8d62f589b8c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.077448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985267-a9a2-43b4-bb3e-5ef281ee39fe None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.086993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.087970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.088431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.088962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.089846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:56.090344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.092077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.102825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 346/688] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:56.108869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:56.114250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.114891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.115182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.115458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.137200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:56.188954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.190437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.226240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284bd0a-a0d2-4f71-b78a-8e9b55a5e6b1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.266054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.267072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.267538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.267992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.281439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.283234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:56.300820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:56.301754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:56.302629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:56.302629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:56.302629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:56.302629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:56.305060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 347/689] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:56.310781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:56.361595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:56.395520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:56.417241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 343/690] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:56.423703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:56.427377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e9a6d-d27e-41f0-a47a-71e81a8aa262 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.468708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.469674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.470236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.470711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.471749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:56.472253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.474440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.489839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 348/691] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:56.496502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:56.547456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe8739-920a-4681-8116-930c9f58981c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.616893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.617918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.618398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.618891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.640058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.641987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:56.668606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:56.669543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:56.670428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:56.670428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:56.670428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:56.670428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:56.673027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6000d-460c-4ad0-aa9d-5405110daee9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.676611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 344/692] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:56.683663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:56.703853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:56.723562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b7747-749f-4ffd-8f85-a302d6b76b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.724206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b7747-749f-4ffd-8f85-a302d6b76b2e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.724480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b7747-749f-4ffd-8f85-a302d6b76b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.724810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b7747-749f-4ffd-8f85-a302d6b76b2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.733666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:56.737489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b7747-749f-4ffd-8f85-a302d6b76b2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.740212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b7747-749f-4ffd-8f85-a302d6b76b2e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ab8b7747-749f-4ffd-8f85-a302d6b76b2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:17:56.758837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:56.759887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:17:56.760827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:56.768002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 349/693] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:17:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:56.768002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc300775-50c3-4178-a49f-eff8b4cd1f60 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.768002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc300775-50c3-4178-a49f-eff8b4cd1f60 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.768256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc300775-50c3-4178-a49f-eff8b4cd1f60 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.769466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 350/694] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:56.770802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc300775-50c3-4178-a49f-eff8b4cd1f60 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:56.820442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88398429-d359-4f1a-a5d6-15ddb77a8785 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.868483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7317795-1ad4-43bb-bb32-b4d1a4bfdd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.869457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317795-1ad4-43bb-bb32-b4d1a4bfdd07 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.869937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317795-1ad4-43bb-bb32-b4d1a4bfdd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.870404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317795-1ad4-43bb-bb32-b4d1a4bfdd07 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.871444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7317795-1ad4-43bb-bb32-b4d1a4bfdd07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:56.871970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7317795-1ad4-43bb-bb32-b4d1a4bfdd07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.873750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7317795-1ad4-43bb-bb32-b4d1a4bfdd07 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.887012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 345/695] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:56.888618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2167bf5-86f0-4c05-b683-4e0c5222914c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.889149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2167bf5-86f0-4c05-b683-4e0c5222914c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.889738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2167bf5-86f0-4c05-b683-4e0c5222914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.910976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e2167bf5-86f0-4c05-b683-4e0c5222914c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:56.912565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e2167bf5-86f0-4c05-b683-4e0c5222914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:56.917947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc300775-50c3-4178-a49f-eff8b4cd1f60 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:56.965821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:56.966446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:56.966722 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:56.967022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:56.968096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:56.968395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:56.970057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:56.981734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.000649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 351/697] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:17:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:57.000973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 346/697] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:17:57.001359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.002402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2167bf5-86f0-4c05-b683-4e0c5222914c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.069977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.131536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2167bf5-86f0-4c05-b683-4e0c5222914c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.134270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802d9b-9b16-4a52-9105-889c2ce132aa None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.170256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:57.171358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:57.171848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:57.172428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:57.178256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318671b9-2448-444a-b428-8d4157602fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:57.179536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318671b9-2448-444a-b428-8d4157602fab None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:57.179816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318671b9-2448-444a-b428-8d4157602fab None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:57.180087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318671b9-2448-444a-b428-8d4157602fab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:57.188687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318671b9-2448-444a-b428-8d4157602fab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:57.190366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318671b9-2448-444a-b428-8d4157602fab None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-318671b9-2448-444a-b428-8d4157602fab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:57.205773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:57.206994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:57.208960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:57.210948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 352/698] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:57.212979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318671b9-2448-444a-b428-8d4157602fab None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.250070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.261943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:57.263458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:57.343075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318671b9-2448-444a-b428-8d4157602fab None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.359377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.389357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 347/699] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:17:57.390517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.396846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:57.397654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:57.397999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:57.398383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:57.410325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:57.411915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:57.430998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:57.431975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:57.432842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:57.432842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:57.432842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:57.432842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:57.437111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 353/700] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:57.438100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.500390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.510188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d94cb-ae81-47ed-a138-bf29cb8ee0f2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.550906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:57.552028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:57.552621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:57.553163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:57.554745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:57.555309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:57.556963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:57.565204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd87ca1-ec09-4dfc-9d5c-2617f04eae86 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.576303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 348/701] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:57.577668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.605622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:57.606341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:57.606546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:57.606823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:57.620789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:57.622289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:17:57.640299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:57.641224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:57.642239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:17:57.642239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:17:57.642239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:17:57.642239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:57.646433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 354/702] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:57.647473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.668920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.694923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23265071-a231-4914-9d03-18f581a5fba6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.730378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:57.731225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:57.731602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:57.731974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:57.732760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:57.733161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:57.734619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:57.752491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 349/703] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:57.753516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.769670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.779577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c51d6-785e-4c2f-95af-846c29defd80 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.827000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:57.827946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:57.828386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:57.828828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:57.829913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=add21760f9ed4b82b942c5dd9a6ed096) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:57.830444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:57.832077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:57.839345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 355/704] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/role_assignments?user.id=add21760f9ed4b82b942c5dd9a6ed096&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:57.840337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:57.950293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a136fb8-3ff7-4bca-93d5-e606493cf1f3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:57.960571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:57.986740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13913dc4-261c-4cf0-a262-9d378f63799b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.019733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0c7ae-2dd5-476a-841a-7b36404022a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.020381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0c7ae-2dd5-476a-841a-7b36404022a5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.020655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0c7ae-2dd5-476a-841a-7b36404022a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.020985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0c7ae-2dd5-476a-841a-7b36404022a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.021610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0c7ae-2dd5-476a-841a-7b36404022a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:58.021909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0c7ae-2dd5-476a-841a-7b36404022a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.023417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0c7ae-2dd5-476a-841a-7b36404022a5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.030656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.031321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.031607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.031884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.032572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.034002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.043243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 350/705] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:58.044873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1420d27-05ea-42a4-ada6-4d9c7af4b9e4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.045390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1420d27-05ea-42a4-ada6-4d9c7af4b9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.045911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1420d27-05ea-42a4-ada6-4d9c7af4b9e4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.052035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 351/706] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:58.054805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08593f20-afc5-4746-9950-aa28f05789f7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.055381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08593f20-afc5-4746-9950-aa28f05789f7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.055882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08593f20-afc5-4746-9950-aa28f05789f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.072199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 356/707] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:17:57 2022] POST /identity/v3/users => generated 254 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:17:58.073132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:58.077078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-08593f20-afc5-4746-9950-aa28f05789f7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:58.078665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-08593f20-afc5-4746-9950-aa28f05789f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:58.197012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 352/708] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:58.212954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecbdfc-23c5-4790-a6bb-7fad3b249532 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.222395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08593f20-afc5-4746-9950-aa28f05789f7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:58.225912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:58.263214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.263872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.264248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.264525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.272960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:58.274985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.277594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:17:58.297447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:58.298375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:17:58.299298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 357/709] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:58.307207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:58.344294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08593f20-afc5-4746-9950-aa28f05789f7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.383011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.383725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.384010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.384283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.384918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.386280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:58.398057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:58.399114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:17:58.399971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:58.402997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 353/710] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:17:58.405342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:58.435700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45666d-6133-4ad2-bd17-88defec14700 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.480330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:58.482733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de9ceb-0f0a-4564-9f52-523b5d947780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.483398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9ceb-0f0a-4564-9f52-523b5d947780 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.483676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9ceb-0f0a-4564-9f52-523b5d947780 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.483950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9ceb-0f0a-4564-9f52-523b5d947780 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.484710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de9ceb-0f0a-4564-9f52-523b5d947780 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:58.485012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de9ceb-0f0a-4564-9f52-523b5d947780 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.486437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de9ceb-0f0a-4564-9f52-523b5d947780 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.500747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 358/711] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:58.501874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c0ee-41e7-4a91-b788-a100361afa34 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.502119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c0ee-41e7-4a91-b788-a100361afa34 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.502390 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c0ee-41e7-4a91-b788-a100361afa34 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.503856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 359/712] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:17:58.510208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3392-c45f-4518-8511-15e46a452335 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.510738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3392-c45f-4518-8511-15e46a452335 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.511267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3392-c45f-4518-8511-15e46a452335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.530771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-291f3392-c45f-4518-8511-15e46a452335 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:17:58.532018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec1972-8a81-4005-9762-6c901c604e80 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.532143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-291f3392-c45f-4518-8511-15e46a452335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:17:58.570887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.571816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.572142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.572412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.573091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:58.573390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.574912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.582916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 354/713] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:17:58.582916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:58.591335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:58.619262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 360/714] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:17:58.620538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f3392-c45f-4518-8511-15e46a452335 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:58.715885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836f7ec-1ae1-4d3e-ade9-1444bd1f9e71 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.747205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:58.754106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.754728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.755070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.755385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.756174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=075e9c0ef8fd406087d05a6ce36dc4e2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:58.756478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.757243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f3392-c45f-4518-8511-15e46a452335 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.758198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.764315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 355/715] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/role_assignments?user.id=075e9c0ef8fd406087d05a6ce36dc4e2&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:58.765171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:58.806308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.808963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.809423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.809847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.810749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:58.811238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.812753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.825377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 361/716] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:58.933478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea64c94-c22f-45d4-8c25-f8293ee31c8d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:58.978754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:58.979458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:58.979738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:58.980014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:58.980658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:58.982124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:17:58.992073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:17:58.993167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:17:58.994612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 356/717] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:17:58.999237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:59.054302 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:59.107549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c90efe-9022-4c6b-b7a2-25b1810a1120 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:59.150443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf6b1-e7cf-4c9c-9bb6-4a0352769f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:59.151095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf6b1-e7cf-4c9c-9bb6-4a0352769f47 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:59.151378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf6b1-e7cf-4c9c-9bb6-4a0352769f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:59.151652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf6b1-e7cf-4c9c-9bb6-4a0352769f47 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:59.152327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf6b1-e7cf-4c9c-9bb6-4a0352769f47 None admin] RBAC: Adding query filter params (name=network) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:59.152674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf6b1-e7cf-4c9c-9bb6-4a0352769f47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:59.154032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf6b1-e7cf-4c9c-9bb6-4a0352769f47 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:59.162713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 362/718] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:17:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:17:59.168271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:17:59.236370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:17:59.291144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8464e33-04fe-4834-a4d6-cee09891c0d5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:17:59.338915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5285ffa-fe26-494f-adee-c394115e7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:17:59.338915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285ffa-fe26-494f-adee-c394115e7bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:17:59.338915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285ffa-fe26-494f-adee-c394115e7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:17:59.338915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285ffa-fe26-494f-adee-c394115e7bf8 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:17:59.339334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5285ffa-fe26-494f-adee-c394115e7bf8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:17:59.339334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5285ffa-fe26-494f-adee-c394115e7bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:17:59.339927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5285ffa-fe26-494f-adee-c394115e7bf8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:17:59.346252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 357/719] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:17:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:00.469790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f9b8-5c35-4aa6-a334-3fa00abe18b1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:00.470333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f9b8-5c35-4aa6-a334-3fa00abe18b1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:00.470333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f9b8-5c35-4aa6-a334-3fa00abe18b1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:00.472483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 363/720] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:00.479265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326ba2-df41-4b61-a1b1-1c5dcb98f8d6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:00.479706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326ba2-df41-4b61-a1b1-1c5dcb98f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:00.479800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326ba2-df41-4b61-a1b1-1c5dcb98f8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:00.498008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0a326ba2-df41-4b61-a1b1-1c5dcb98f8d6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:00.499224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0a326ba2-df41-4b61-a1b1-1c5dcb98f8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:00.591669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 358/721] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 14:18:00.625177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273f9b8-5c35-4aa6-a334-3fa00abe18b1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:00.752053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273f9b8-5c35-4aa6-a334-3fa00abe18b1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:00.766767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:00.799169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27340313-c07a-4a48-b202-e6f1fa0c5f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:00.800076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27340313-c07a-4a48-b202-e6f1fa0c5f5e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:00.800357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27340313-c07a-4a48-b202-e6f1fa0c5f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:00.800640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27340313-c07a-4a48-b202-e6f1fa0c5f5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:00.818795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27340313-c07a-4a48-b202-e6f1fa0c5f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-27340313-c07a-4a48-b202-e6f1fa0c5f5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:00.825658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:00.826595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:00.828080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 364/722] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:00.832860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a326ba2-df41-4b61-a1b1-1c5dcb98f8d6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:00.900475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b13a0-f9b0-4f8f-990d-cfe90e069710 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:00.900819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b13a0-f9b0-4f8f-990d-cfe90e069710 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:00.901023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b13a0-f9b0-4f8f-990d-cfe90e069710 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:00.902419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 365/723] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:00.907460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc07c5-75fc-445c-bfcb-74e795e1915f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:00.907739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc07c5-75fc-445c-bfcb-74e795e1915f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:00.908014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc07c5-75fc-445c-bfcb-74e795e1915f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:00.915543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:00.935052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-16bc07c5-75fc-445c-bfcb-74e795e1915f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:00.936269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-16bc07c5-75fc-445c-bfcb-74e795e1915f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:00.998825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a326ba2-df41-4b61-a1b1-1c5dcb98f8d6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.068685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.069815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.071093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.072205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.074646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:01.075183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.077096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:01.080770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 366/724] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:18:01.089233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 359/725] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:01.095104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc07c5-75fc-445c-bfcb-74e795e1915f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.106908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.224042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc07c5-75fc-445c-bfcb-74e795e1915f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.227411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:01.267031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868b914-1c29-4a6f-acc9-9e915e31ecca None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.277298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:01.281421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.282367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.282891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.283374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.296175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.298020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:01.314087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:01.315006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:01.315880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:01.315880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:01.315880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:01.315880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:01.318138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 367/726] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:01.323413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.332927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1a14f-853a-470e-ad48-d3242808b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.333861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a14f-853a-470e-ad48-d3242808b58b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.335317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a14f-853a-470e-ad48-d3242808b58b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.335805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a14f-853a-470e-ad48-d3242808b58b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.349034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1a14f-853a-470e-ad48-d3242808b58b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2ae1a14f-853a-470e-ad48-d3242808b58b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:01.355202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:01.356102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:01.356920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 360/727] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:01.358088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd3e79-95bb-4cac-9328-317cafb9753c None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.358088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd3e79-95bb-4cac-9328-317cafb9753c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.358334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd3e79-95bb-4cac-9328-317cafb9753c None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.359829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 361/728] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:01.364606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd3e79-95bb-4cac-9328-317cafb9753c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.440572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c4f4e-fcc1-46be-a693-0c5a750e553f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.471438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd3e79-95bb-4cac-9328-317cafb9753c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.477508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f8839-fdc1-42db-b8ad-d4ee0ddf3e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.478295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f8839-fdc1-42db-b8ad-d4ee0ddf3e77 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.478672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f8839-fdc1-42db-b8ad-d4ee0ddf3e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.479017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f8839-fdc1-42db-b8ad-d4ee0ddf3e77 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.479756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f8839-fdc1-42db-b8ad-d4ee0ddf3e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:01.480064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f8839-fdc1-42db-b8ad-d4ee0ddf3e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.481538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f8839-fdc1-42db-b8ad-d4ee0ddf3e77 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:01.497569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 368/729] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:01.497569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bcfd2-6613-4ada-aa79-dd5c985408b9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.497569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bcfd2-6613-4ada-aa79-dd5c985408b9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.497569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bcfd2-6613-4ada-aa79-dd5c985408b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.511372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8cb11-9944-40dc-8edd-9a2840c17bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.512093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8cb11-9944-40dc-8edd-9a2840c17bea None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.512342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ea1bcfd2-6613-4ada-aa79-dd5c985408b9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:01.512443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8cb11-9944-40dc-8edd-9a2840c17bea None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.512637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8cb11-9944-40dc-8edd-9a2840c17bea None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.513341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8cb11-9944-40dc-8edd-9a2840c17bea None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:01.513493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ea1bcfd2-6613-4ada-aa79-dd5c985408b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:01.513650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8cb11-9944-40dc-8edd-9a2840c17bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.515010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8cb11-9944-40dc-8edd-9a2840c17bea None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:01.524517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:01.525807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 362/730] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:01.527103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e12b-0958-4126-8583-edf27a532bd2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.527386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e12b-0958-4126-8583-edf27a532bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.527661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e12b-0958-4126-8583-edf27a532bd2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.532750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 363/731] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:01.532750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84e12b-0958-4126-8583-edf27a532bd2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.569081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:01.595090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 369/732] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:18:01.595992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bcfd2-6613-4ada-aa79-dd5c985408b9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.655351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84e12b-0958-4126-8583-edf27a532bd2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.692483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5b25-2fea-4673-94c1-1da56239b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.693122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5b25-2fea-4673-94c1-1da56239b362 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.693399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5b25-2fea-4673-94c1-1da56239b362 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.693659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5b25-2fea-4673-94c1-1da56239b362 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.701346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5b25-2fea-4673-94c1-1da56239b362 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.702772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5b25-2fea-4673-94c1-1da56239b362 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:01.705836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bcfd2-6613-4ada-aa79-dd5c985408b9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-277c5b25-2fea-4673-94c1-1da56239b362 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:01.714861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:01.716108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:01.717060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 364/733] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:01.717690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656445-91d4-4628-b616-3eab59d97532 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.717759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656445-91d4-4628-b616-3eab59d97532 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.717999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656445-91d4-4628-b616-3eab59d97532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.736654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-75656445-91d4-4628-b616-3eab59d97532 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:01.737825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-75656445-91d4-4628-b616-3eab59d97532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:01.748350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.748992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.749276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.749547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.760974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.762695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:01.769090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:01.778491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:01.779599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:01.780501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:01.780501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:01.780501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:01.780501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:01.780501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 370/734] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:01.780501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.818957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 365/735] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:18:01.818957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75656445-91d4-4628-b616-3eab59d97532 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.861727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:01.900123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae23a4-da73-4f49-b60b-757bac191e8a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.934228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75656445-91d4-4628-b616-3eab59d97532 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:01.936091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.936827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.937104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.937375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.938053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.939438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:01.966912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 371/736] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:01 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:01.966912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:01.988982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:01.989857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:01.990204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:01.990673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:01.992266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:01.992643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:01.994609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.016206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 366/737] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:02.017737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.023872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.079086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca25a-00fe-4d40-9c74-1f9194860e18 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.118204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.121295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.121948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.122786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.123103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.124019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:02.124334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.125721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.138881 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b550ec-aa82-4c01-965c-bdcb9ef709d7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.140112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 372/738] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:02.140646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.182027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b273ae-0d0f-4e8c-9eb4-846498e06688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.182670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b273ae-0d0f-4e8c-9eb4-846498e06688 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.182978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b273ae-0d0f-4e8c-9eb4-846498e06688 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.183262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b273ae-0d0f-4e8c-9eb4-846498e06688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.195592 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b273ae-0d0f-4e8c-9eb4-846498e06688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-98b273ae-0d0f-4e8c-9eb4-846498e06688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:02.200587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:02.201655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:02.202606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 367/739] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:02.203421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e19bc-1d5d-4d90-bb51-b019a771f968 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.203561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e19bc-1d5d-4d90-bb51-b019a771f968 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.203843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e19bc-1d5d-4d90-bb51-b019a771f968 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.208900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 368/740] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:02.208900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e19bc-1d5d-4d90-bb51-b019a771f968 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.261658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa09320-7ae1-49b3-a224-716141e222a6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.280289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.298703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a156d8-6e87-425d-8fb8-477ab49ee838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.299360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a156d8-6e87-425d-8fb8-477ab49ee838 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.299638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a156d8-6e87-425d-8fb8-477ab49ee838 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.300163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a156d8-6e87-425d-8fb8-477ab49ee838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.317977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e19bc-1d5d-4d90-bb51-b019a771f968 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.355464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.356109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.356387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.356661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.363959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.365391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.368577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.373837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a156d8-6e87-425d-8fb8-477ab49ee838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.375313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a156d8-6e87-425d-8fb8-477ab49ee838 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:02.378475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:02.379476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 369/741] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:02.380902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.485368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c93e-18be-4546-a948-22b3959013eb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.490283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 373/742] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 14:18:02.490283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e257459-acc0-42a9-8c10-79ebd4a701c5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.490283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e257459-acc0-42a9-8c10-79ebd4a701c5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.490283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e257459-acc0-42a9-8c10-79ebd4a701c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.507151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9e257459-acc0-42a9-8c10-79ebd4a701c5 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:02.508351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9e257459-acc0-42a9-8c10-79ebd4a701c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:02.523236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.527475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.528282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.528572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.528852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.529659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:02.529990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.531378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.541884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 370/743] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:02.542523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.588310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 374/744] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 14:18:02.589148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e257459-acc0-42a9-8c10-79ebd4a701c5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.620458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.665505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80238143-c5a0-4b39-ad22-8a57278e33c1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.703014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.703219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e257459-acc0-42a9-8c10-79ebd4a701c5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.703639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.703922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.704198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.705111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:02.705537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.706991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.729921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 371/745] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:02.729921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.746299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.746994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.747292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.747557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.759174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.760606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.769829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:02.777753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:02.778694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:02.779621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:02.779621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:02.779621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:02.779621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:02.786856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 375/746] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:02.787658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.837274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02ea72-5938-44ca-985a-043d9e68c99e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.858510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:02.873870 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.874452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.874734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.875041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.885624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.887032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:02.901580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:02.902462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:02.903386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:02.903386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:02.903386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:02.903386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:02.903386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 372/747] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:02.903665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:02.913527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e3f0d-62eb-4cb8-a614-3fa7e64cdcb6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:02.950514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:02.951341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:02.951451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:02.951703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:02.952388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:02.953736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:02.969224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 376/748] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:02 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:02.970480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.036100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785d156-269d-4b37-95de-6b3c7db9c424 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.053991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:03.106563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.107407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.107534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.107810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.119345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b4ecd-b2da-43e5-835c-d6f040e6558b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.150495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:03.188215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.189019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.189406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.189928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.191310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:03.191743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.193187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.193298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.194648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.206710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 377/749] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:03.207549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.314410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a262dc0-5bd8-4f8b-b731-2e395aae4dae None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.317260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 373/750] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:03.318313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.335087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:03.350740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.351391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.351717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.352055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.352668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:03.352975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.354338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.366638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 378/751] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:03.367485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.393333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:03.433247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684c5a3-5d40-4a24-aa40-5716b2dd7c7c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.470004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.470996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.471537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7dc38-fc29-479f-bf99-319af885a56e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.471840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.472430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.481848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.483746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:03.497887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:03.499048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:03.500092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:03.506731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 374/752] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:03.508021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.510895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.511540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.511814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.512194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.523854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.525295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:03.540925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:03.542045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:03.543147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:03.543147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:03.543147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:03.543147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:03.543147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 379/753] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:03.543147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.575810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:03.622610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb96d-1439-4007-8508-4f9b845d8d6a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.651524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:03.653714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa2f4a-ffeb-4561-8a46-d082fe9470c6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.660822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951421a-94ae-445a-bcef-911091963726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.662058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951421a-94ae-445a-bcef-911091963726 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.663458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951421a-94ae-445a-bcef-911091963726 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.663931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951421a-94ae-445a-bcef-911091963726 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.672421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951421a-94ae-445a-bcef-911091963726 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.674162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951421a-94ae-445a-bcef-911091963726 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5951421a-94ae-445a-bcef-911091963726 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:03.686784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:03.687981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:03.689152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:03.690485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 375/754] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:03.691540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951421a-94ae-445a-bcef-911091963726 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.696460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.697312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.697863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.698309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.699272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:03.699749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.701323 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.715598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 380/755] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:03.716781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.820090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951421a-94ae-445a-bcef-911091963726 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.832991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:03.852123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bced761-b107-4277-a984-7e0e86aa7b5c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:03.867895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.868517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.868798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.869071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.869756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:03.870055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.871455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.884387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 376/756] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:03.885227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.901256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:03.902668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:03.903411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:03.903883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:03.905473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:03.905979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:03.907628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:03.930645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 381/757] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:03.931611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:03.947653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:04.006036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f4d9e-ce0d-4ec2-81db-916259ca0b15 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.043486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.044225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.044503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.044797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.045657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=add21760f9ed4b82b942c5dd9a6ed096) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:04.045967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.047616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.054043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e8959-6c22-46b1-ab15-d7845c5738d4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.062667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 377/758] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/role_assignments?user.id=add21760f9ed4b82b942c5dd9a6ed096&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:04.064642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.093955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec91db-a7b6-46da-be53-b3bad245810f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.094674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec91db-a7b6-46da-be53-b3bad245810f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.094992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec91db-a7b6-46da-be53-b3bad245810f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.095267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec91db-a7b6-46da-be53-b3bad245810f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.103882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec91db-a7b6-46da-be53-b3bad245810f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.105411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec91db-a7b6-46da-be53-b3bad245810f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4cec91db-a7b6-46da-be53-b3bad245810f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:04.118421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:04.119421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 382/759] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:04.121267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758c4a-8188-45d2-a791-999eb69f197d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.122269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758c4a-8188-45d2-a791-999eb69f197d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.122269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758c4a-8188-45d2-a791-999eb69f197d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.128665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 383/760] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:04.128665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6758c4a-8188-45d2-a791-999eb69f197d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.144689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:04.205038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec6631-86b9-4ee9-8751-8f9899f4d9f9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.222513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:04.253476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926124a5-1d6e-4561-86e2-005360e493b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.254554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926124a5-1d6e-4561-86e2-005360e493b9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.254840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926124a5-1d6e-4561-86e2-005360e493b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.255157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926124a5-1d6e-4561-86e2-005360e493b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.268052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6758c4a-8188-45d2-a791-999eb69f197d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.269304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926124a5-1d6e-4561-86e2-005360e493b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.271590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926124a5-1d6e-4561-86e2-005360e493b9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-926124a5-1d6e-4561-86e2-005360e493b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:04.292920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:04.293820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:04.294691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:04.294691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:04.294691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:04.294691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:04.295436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 378/761] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:04.296172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027d1bd-f8db-4c91-913d-b740e84bad93 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.296661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027d1bd-f8db-4c91-913d-b740e84bad93 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.296759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027d1bd-f8db-4c91-913d-b740e84bad93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.310419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.311069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.311353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.311629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.312400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:04.312707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.314127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.318045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e027d1bd-f8db-4c91-913d-b740e84bad93 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:04.319911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e027d1bd-f8db-4c91-913d-b740e84bad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:04.327580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 384/762] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:04.328527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.404643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:04.409926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 379/763] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:04.411270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027d1bd-f8db-4c91-913d-b740e84bad93 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.439620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6999fa-f56c-4113-9520-9acfc272888a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.480154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.480789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.481180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.481534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.482299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:04.482668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.484130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.501699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 385/764] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:04.502614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.538337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:04.546756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027d1bd-f8db-4c91-913d-b740e84bad93 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.585545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.586171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.586490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.586771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.587595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=5f26e831b05740c9a1d510f107e74b57) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:04.587913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.589345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.595360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 380/765] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/role_assignments?user.id=5f26e831b05740c9a1d510f107e74b57&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:04.596273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.656131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a868fc-afac-427c-a217-805d261d0678 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.727458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:04.738762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.739369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.739650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.739927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.740568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.741997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:04.767770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:04.768924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:04.770006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 386/766] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:04.776001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.780416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad4231-4e9d-401b-987f-a6fb90630235 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.837461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.838688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.839287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.839579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.858145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.863779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:04.900829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:04.901945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:04.903028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:04.904758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 381/767] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:04.917392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:04.941232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b76009-e533-4186-aa3f-01b28d10a659 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:04.959265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:04.981637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:04.982261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:04.982540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:04.982813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:04.983517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:04.983823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:04.985190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:04.996761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 387/768] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:05.002240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:05.113529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:05.118033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40bc7-5d30-42bd-8a5d-99f4f02dffee None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:05.193641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8171-db8e-4fa4-97ff-14c59a46b1c4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:05.196238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:05.196875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.197158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.197435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.198195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:05.198505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:05.199991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:05.223093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 382/769] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:05.235952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:05.251484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:05.252052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.252343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.252676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.253373 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:05.253686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:05.255092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:05.261310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 388/770] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Aug 6 14:18:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:05.266705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:05.353462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:05.377048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715ff7b-50a8-41a9-a0e6-3fb62259b1ec None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:05.409178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3292-df90-4920-96bd-b193a75f149d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:05.413384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64496e20-2f2c-4a0c-b6c8-4b5c90604c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:05.414059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64496e20-2f2c-4a0c-b6c8-4b5c90604c35 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.414341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64496e20-2f2c-4a0c-b6c8-4b5c90604c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.414614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64496e20-2f2c-4a0c-b6c8-4b5c90604c35 None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/075e9c0ef8fd406087d05a6ce36dc4e2/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.416632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64496e20-2f2c-4a0c-b6c8-4b5c90604c35 None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=075e9c0ef8fd406087d05a6ce36dc4e2, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:05.418071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64496e20-2f2c-4a0c-b6c8-4b5c90604c35 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:05.439314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 383/771] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:18:05 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/075e9c0ef8fd406087d05a6ce36dc4e2/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:18:05.439314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21671d97-7334-493d-970b-9767243e6813 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.439314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21671d97-7334-493d-970b-9767243e6813 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.439314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21671d97-7334-493d-970b-9767243e6813 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.439314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 384/772] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:05.444510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753885b-71bb-4015-8a54-c178fa27ebda None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.444790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753885b-71bb-4015-8a54-c178fa27ebda None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.445062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753885b-71bb-4015-8a54-c178fa27ebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.455256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:05.468031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6753885b-71bb-4015-8a54-c178fa27ebda None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:05.469714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6753885b-71bb-4015-8a54-c178fa27ebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:05.473895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:05.474744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.475764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.476193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.477184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] RBAC: Adding query filter params (interface=public, service_id=355354cdc0634a11a25201c57c7b4ef6, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:05.477648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:05.485700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:05.502906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 389/773] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:18:05 2022] GET /identity/v3/endpoints?service_id=355354cdc0634a11a25201c57c7b4ef6&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:05.508102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:05.549353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 385/774] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:05.575691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753885b-71bb-4015-8a54-c178fa27ebda None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:05.654294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:05.696791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753885b-71bb-4015-8a54-c178fa27ebda None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:05.730230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0e1b0-a1c8-4435-8847-bbd3627e3e8f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:05.746239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:05.746994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.747273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.747544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] PATH_INFO: `/v3/services/network` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.748174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:05.749540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:05.760384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:05.761388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:18:05.762238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:05.769616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 386/775] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:05 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:05.771200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:05.781013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:05.781638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.781913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.782181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.782828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:05.784212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:05.793473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 390/776] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:05 2022] GET /identity/v3/services => generated 772 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:05.938089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950b5eb-bc17-44e3-b0de-e794ba13369e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:05.985524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:05.991281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:05.992175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:05.992632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:05.993156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:05.993840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] RBAC: Adding query filter params (name=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:05.994147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:05.995553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:06.004836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 387/777] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:06.006370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:06.063033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:06.120292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5971368-9b23-456f-82a0-c6abeeee5735 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:06.156241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1274f0-0c75-4b4b-b726-7b2388d20f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:06.156869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1274f0-0c75-4b4b-b726-7b2388d20f3e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.157145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1274f0-0c75-4b4b-b726-7b2388d20f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.157417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1274f0-0c75-4b4b-b726-7b2388d20f3e None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.158087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1274f0-0c75-4b4b-b726-7b2388d20f3e None admin] RBAC: Adding query filter params (type=network) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:06.158388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1274f0-0c75-4b4b-b726-7b2388d20f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:06.159770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1274f0-0c75-4b4b-b726-7b2388d20f3e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:06.165748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 391/778] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:06.171586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:06.271338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253436ef-4bed-4439-b2fe-0635d96b4172 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.271617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253436ef-4bed-4439-b2fe-0635d96b4172 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.272513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253436ef-4bed-4439-b2fe-0635d96b4172 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.274012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 392/779] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:06.279062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce480bb-a6a4-4959-9bb8-f76ab4f200c3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.279348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce480bb-a6a4-4959-9bb8-f76ab4f200c3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.279625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce480bb-a6a4-4959-9bb8-f76ab4f200c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.307595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5ce480bb-a6a4-4959-9bb8-f76ab4f200c3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:06.308789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5ce480bb-a6a4-4959-9bb8-f76ab4f200c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:06.334040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acacbc-9f42-4bcf-9d3c-0287debe16eb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:06.396510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:06.397777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.398052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.398318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.399092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] RBAC: Adding query filter params (interface=public, service_id=830aa91665174f2397083dd0619e930d, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:06.399400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:06.400769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:06.413641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 388/780] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:18:06 2022] GET /identity/v3/endpoints?service_id=830aa91665174f2397083dd0619e930d&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:06.417948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:06.437070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 393/781] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:06.497776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce480bb-a6a4-4959-9bb8-f76ab4f200c3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:06.512182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:06.567762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:06.572674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d2369-4f53-483c-b85a-d2ac30350c43 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:06.612544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:06.613524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.613906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.614286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.615073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:06.617077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:06.627386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 389/782] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:06 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:06.681732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce480bb-a6a4-4959-9bb8-f76ab4f200c3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:06.737094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b62052-3d08-4306-930a-6dc7e6789f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:06.737737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62052-3d08-4306-930a-6dc7e6789f34 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.738018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62052-3d08-4306-930a-6dc7e6789f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.738289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62052-3d08-4306-930a-6dc7e6789f34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.754637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b62052-3d08-4306-930a-6dc7e6789f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-62b62052-3d08-4306-930a-6dc7e6789f34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:06.760530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:06.761541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:06.762399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 394/783] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:06.768193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:06.890763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e146df-ab22-499a-9c97-27db3a616f8a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:06.895699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea0c80-93d3-4a77-a744-1e956260635c None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.895982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea0c80-93d3-4a77-a744-1e956260635c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.896252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea0c80-93d3-4a77-a744-1e956260635c None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.898224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 395/784] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:06.903396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305439-63ba-4734-a873-26cbb5ff35ba None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.903904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305439-63ba-4734-a873-26cbb5ff35ba None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.904388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305439-63ba-4734-a873-26cbb5ff35ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.926523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-04305439-63ba-4734-a873-26cbb5ff35ba None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:06.927260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:06.929083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04305439-63ba-4734-a873-26cbb5ff35ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:06.932236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:06.933070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:06.933545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:06.933976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:06.934890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:06.935418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:06.936964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:06.947294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 390/785] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:06.953689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.010995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 396/786] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:07.027926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04305439-63ba-4734-a873-26cbb5ff35ba None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.063600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a5ece-9cee-4f1f-9d76-825529045904 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:07.064166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:07.101683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:07.102781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:07.103301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:07.103784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:07.115333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:07.117893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:07.135038 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04305439-63ba-4734-a873-26cbb5ff35ba None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:07.135660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:07.136484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:07.137372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:07.137372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:07.137372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:07.137372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:07.140414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 391/787] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:07.147865 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.178893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:07.178893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:07.178893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:07.178893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:07.194919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:07.243269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:07.328005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4252-b0e0-4f04-bd67-e336e784cb20 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:07.343444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:07.344439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:07.345254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 397/788] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:18:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:07.351389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.371432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:07.372414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:07.372896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:07.373370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:07.374287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:07.375521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:07.377268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:07.409063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 392/789] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:07.415069 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.528711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:07.542428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a991c2-5062-4b58-b029-acdf7903b793 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:07.599253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:07.602297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bb75b-4058-4bc4-8e4b-9ce8d943285f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:07.612785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:07.613411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:07.613685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:07.613956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:07.614667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:07.614993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:07.616878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:07.637280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 398/790] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:18:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:07.641422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.665671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:07.666236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:07.666518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:07.666796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:07.675005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:07.676644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:07.689171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:07.690361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:18:07.691565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:07.693800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 393/791] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:07.700126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.775657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c5fce-75ca-4ad6-9c6e-ab5e33c02515 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:07.812498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:07.813408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:07.813835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:07.814253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:07.825126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:07.826832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:07.842262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:07.843188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:07.844092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:07.844092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:07.844092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:07.844092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:07.850971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 399/792] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:07.852972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:07.873678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:07.918719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b407d11-1d87-4312-856c-3330f208c25a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:07.991588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:07.992217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:07.992488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:07.992760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:07.993563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:07.993868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:07.995336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.012262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65005f9a-a69d-414a-bfc0-4012a26d0c26 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.026108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 394/793] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Aug 6 14:18:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:08.031736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.049075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.049951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.050241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.050517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.051246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:08.051550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.052907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.070556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 400/794] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:08.072854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.079494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.153353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.192281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aff26c-2d10-4e74-a78e-d18c2c977c72 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.229623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde33abf-0b20-4755-a474-a35726e15ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.230266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33abf-0b20-4755-a474-a35726e15ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.230545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33abf-0b20-4755-a474-a35726e15ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.230819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33abf-0b20-4755-a474-a35726e15ea1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.238783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde33abf-0b20-4755-a474-a35726e15ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.240322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde33abf-0b20-4755-a474-a35726e15ea1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bde33abf-0b20-4755-a474-a35726e15ea1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:08.252761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:08.253764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:08.254696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 401/795] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:08.255796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f956a-0171-4ce0-ae75-3ba694443fa0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.255796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f956a-0171-4ce0-ae75-3ba694443fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.255947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f956a-0171-4ce0-ae75-3ba694443fa0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.257445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 402/796] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:08.262554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7cca8-fbb9-4e86-869d-512ef99379c1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.263095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7cca8-fbb9-4e86-869d-512ef99379c1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.263568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7cca8-fbb9-4e86-869d-512ef99379c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.279415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e74c9-2f80-4062-9848-fb9ece2cd611 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.285308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c4e7cca8-fbb9-4e86-869d-512ef99379c1 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:08.287223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c4e7cca8-fbb9-4e86-869d-512ef99379c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:08.323309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.324926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.325405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.326135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.339639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.341790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:08.362539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:08.363497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:08.364364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:08.364364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:08.364364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:08.364364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:08.364590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 395/797] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:08.370261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.382913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 403/798] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:08.382913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7cca8-fbb9-4e86-869d-512ef99379c1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.447426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.448677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.486095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639b6e-0b20-4d43-a8ce-11c7416b0dd1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.491899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7cca8-fbb9-4e86-869d-512ef99379c1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.523439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748c704-7e1d-4141-8d1b-9cb37c6e649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.524061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748c704-7e1d-4141-8d1b-9cb37c6e649d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.524339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748c704-7e1d-4141-8d1b-9cb37c6e649d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.524617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748c704-7e1d-4141-8d1b-9cb37c6e649d None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.525402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748c704-7e1d-4141-8d1b-9cb37c6e649d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:08.525711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748c704-7e1d-4141-8d1b-9cb37c6e649d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.527162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748c704-7e1d-4141-8d1b-9cb37c6e649d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.531120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.531908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.532332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.532765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.533593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:08.534101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.536051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.545245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 396/799] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:08.546363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865691c-5e3a-4025-a3d4-04395290ef95 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.546655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865691c-5e3a-4025-a3d4-04395290ef95 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.546956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865691c-5e3a-4025-a3d4-04395290ef95 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.548162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 397/800] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:08.550982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865691c-5e3a-4025-a3d4-04395290ef95 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.556994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 404/801] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:08.557951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.666947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2af54-25d5-4508-b0b2-a200ce45213c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.674366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865691c-5e3a-4025-a3d4-04395290ef95 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.686858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.702811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.707489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.708218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.708501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.708766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.713003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31ed3f-0c80-40a5-85e6-b6f7e0386c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.713675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31ed3f-0c80-40a5-85e6-b6f7e0386c43 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.713957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31ed3f-0c80-40a5-85e6-b6f7e0386c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.714230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31ed3f-0c80-40a5-85e6-b6f7e0386c43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.719806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.721219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.725676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31ed3f-0c80-40a5-85e6-b6f7e0386c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-ee31ed3f-0c80-40a5-85e6-b6f7e0386c43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:08.730694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:08.731898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:08.732767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 398/802] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:08.733470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610d353-2db8-4528-a184-38c1d7e2e258 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.733470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610d353-2db8-4528-a184-38c1d7e2e258 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.734146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610d353-2db8-4528-a184-38c1d7e2e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:08.741837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:08.742735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:08.743668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:08.743668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:08.743668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:08.743668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:08.743668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 405/803] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:08.744276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.754131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5610d353-2db8-4528-a184-38c1d7e2e258 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:08.755351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5610d353-2db8-4528-a184-38c1d7e2e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:08.827068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 399/804] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:08.827976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610d353-2db8-4528-a184-38c1d7e2e258 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.863969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeecc07-156c-460a-8650-4b393e7d9ece None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.902262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.902933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.903215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.903484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.910229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.911673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:08.923187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:08.924099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:08.925224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 406/805] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:08.926195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:08.951359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610d353-2db8-4528-a184-38c1d7e2e258 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:08.961144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.968422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:08.986141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:08.986768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:08.987082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:08.987358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:08.988118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:08.988432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:08.989800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:08.998382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 400/806] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:08.999189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.037194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c08383-374d-4579-bed7-87e1ff64b0d9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.071251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.071871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.072147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.072423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.073101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:09.073405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.074744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.086157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 407/807] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:09.086956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.095561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542bd56-6fe9-45b3-b855-ca6af5afdba9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.129240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0398c-0c1e-4871-84a6-d9fbf0dd5074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.129929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0398c-0c1e-4871-84a6-d9fbf0dd5074 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.130212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0398c-0c1e-4871-84a6-d9fbf0dd5074 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.130518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0398c-0c1e-4871-84a6-d9fbf0dd5074 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.131252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0398c-0c1e-4871-84a6-d9fbf0dd5074 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.133108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0398c-0c1e-4871-84a6-d9fbf0dd5074 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-c7f0398c-0c1e-4871-84a6-d9fbf0dd5074 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:09.141799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:09.142828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:09.143693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 401/808] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Aug 6 14:18:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:09.144614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1d32c-799f-43d1-98b8-d8fb3dabc6c0 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.144614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1d32c-799f-43d1-98b8-d8fb3dabc6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.144734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1d32c-799f-43d1-98b8-d8fb3dabc6c0 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.145878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 402/809] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:09.146566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1d32c-799f-43d1-98b8-d8fb3dabc6c0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.182055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.191971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f2edc-64bc-4193-b297-58864f051f5d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.193297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.225897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.226452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.227041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.227041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.227774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:09.228079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.229485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.240693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 408/810] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:09.241580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.251374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1d32c-799f-43d1-98b8-d8fb3dabc6c0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.285964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.286604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.286902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.287194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.296916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.298352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:09.312858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:09.313737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:09.314964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:09.314964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:09.314964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:09.314964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:09.314964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 403/811] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:09.314964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.343880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c44a9-e2b8-43a9-b9d8-864c596bb8d3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.376238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142266e2-dd3b-4f46-9d01-7fe9a3a6b946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.376871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142266e2-dd3b-4f46-9d01-7fe9a3a6b946 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.377154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142266e2-dd3b-4f46-9d01-7fe9a3a6b946 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.377430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142266e2-dd3b-4f46-9d01-7fe9a3a6b946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.384487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142266e2-dd3b-4f46-9d01-7fe9a3a6b946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.385896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142266e2-dd3b-4f46-9d01-7fe9a3a6b946 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-142266e2-dd3b-4f46-9d01-7fe9a3a6b946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:09.397550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:09.398570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:09.399491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 409/812] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:09.399984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0110840-a875-4d36-a139-37f0d358f04d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.400053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0110840-a875-4d36-a139-37f0d358f04d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.400331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0110840-a875-4d36-a139-37f0d358f04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.411712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.417448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f0110840-a875-4d36-a139-37f0d358f04d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:09.418630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f0110840-a875-4d36-a139-37f0d358f04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:09.419464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be70c7-3d8c-4e49-b3a8-883a00995481 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.421378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.453648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.454258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.454537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.454805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.455498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:09.455798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.457142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.462047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 404/813] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:09.462781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.489101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 410/814] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:09.489862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0110840-a875-4d36-a139-37f0d358f04d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.560418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89922-4a15-40e3-8948-662004bf551c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.593019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.593186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0110840-a875-4d36-a139-37f0d358f04d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.593688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.593968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.594245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/add21760f9ed4b82b942c5dd9a6ed096/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.596213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=add21760f9ed4b82b942c5dd9a6ed096, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.597631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.609741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 405/815] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:18:09 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/add21760f9ed4b82b942c5dd9a6ed096/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:18:09.610530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.629652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.630291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.630571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.630840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.632249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:09.632555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.633957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.644013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.646069 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.654225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 411/816] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:09.655217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.739018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea6d2b-401d-4321-bb55-50d659155a3e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.774749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0fceb-90dd-4aab-8d75-09f774107177 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.780667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.781289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.781567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.781838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.782586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:09.782909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.784334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.797381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 406/817] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:09.798100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.814646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.815310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.815593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.815851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.816522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:09.816827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.818203 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.824340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 412/818] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:09.825052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.886658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.896355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:09.911912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb65d12-c4a0-433e-807c-d9d4e41afe4c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.937376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b368e5e-b2f5-400a-911a-c498af0e409f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:09.955375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.956021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.956298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.956570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] PATH_INFO: `/v3/services/network` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.957202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.959521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:09.970318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:09.976911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:09.978040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 407/819] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:09.979014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:09.980453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:09.981242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:09.981622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:09.982012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:09.991587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:09.993124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:10.005124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:10.006280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:10.007529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:10.011208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 413/820] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:18:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:10.011827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:10.082890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea63117-216a-4986-9a21-2f2331fcc38b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:10.114206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4d625-4ec8-483d-a6c5-eee4e2035e2b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:10.121768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:10.122448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:10.122726 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:10.123032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/5f26e831b05740c9a1d510f107e74b57/roles/c5d40ea25aa5440d8170744359aaa18a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:10.135880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:10.137798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:10.139736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=5f26e831b05740c9a1d510f107e74b57, role_id=c5d40ea25aa5440d8170744359aaa18a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:10.141170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:10.149625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:10.150253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:10.150522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:10.150796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:10.151549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:10.152892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:10.166898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 408/821] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:18:09 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/5f26e831b05740c9a1d510f107e74b57/roles/c5d40ea25aa5440d8170744359aaa18a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:18:10.166898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:10.196174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 414/822] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:10.197180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:10.379380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fe553-36d8-4437-8e39-6c53c1095245 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:10.410858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf9741-ac44-4f87-95a4-12f74e0e1e49 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:10.423737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:10.425109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:10.425407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:10.425685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:10.426367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] RBAC: Adding query filter params (name=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:10.426674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:10.428516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:10.435004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 409/823] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:10.440941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:10.456746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:10.457959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:10.458404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:10.459129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:10.461104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:10.461565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:10.463707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:10.489433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 415/824] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:18:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:10.495069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:10.518681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:10.555392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:10.608706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b60ec-0eab-45a5-b787-f210512b9e6c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:10.646044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b85fe-67d4-4556-aca7-8b862864ad82 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:10.674598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:10.675499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:10.675775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:10.676045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:10.676721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] RBAC: Adding query filter params (type=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:10.678405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:10.679815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:10.691770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 410/825] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:10.699974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:10.718117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:10.718735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:10.719037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:10.719308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:10.847057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:10.849024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:10.886960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:10.914299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3375e4-f878-4129-ad85-9795987bd286 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:10.964966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 416/826] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:10.966328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:10.971409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:11.007557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:11.008191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.008473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.008750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.009478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:11.011480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:11.081476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ee1e8-230e-41bb-a12b-9821ee5770c5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:11.086623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 411/827] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:18:11.138484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:11.148032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:11.148704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.149049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.149354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.163410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:11.165041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:11.186910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:11.187865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:11.188779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:11.188779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:11.188779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:11.188779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:11.192392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 417/828] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:11.197990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:11.345447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ffca5-bbbb-43f8-9377-fc513542ba84 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:11.381301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:11.382289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.382785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.383321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.384210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:11.384714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:11.386269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:11.399276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 412/829] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:11.404945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:11.524020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92f0c-98b3-4634-a1e4-dcaee614e725 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:11.561082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:11.561728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.562005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.562284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.570813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:11.572287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:11.587213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:11.588198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:11.589212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 418/830] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:11.594106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:11.645878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:11.714636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8888cb7-00d9-47ef-be98-d02e28ad11ae None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:11.748097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6293a4-3d48-49d8-9527-9878739a4afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:11.748721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6293a4-3d48-49d8-9527-9878739a4afa None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.749003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6293a4-3d48-49d8-9527-9878739a4afa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.749278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6293a4-3d48-49d8-9527-9878739a4afa None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.750023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6293a4-3d48-49d8-9527-9878739a4afa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:11.750330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6293a4-3d48-49d8-9527-9878739a4afa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:11.751839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6293a4-3d48-49d8-9527-9878739a4afa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:11.764323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 413/831] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:11.770058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:11.815755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:11.840998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95aba8-5ec2-49a5-9ada-ff0ab33f7550 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.841285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95aba8-5ec2-49a5-9ada-ff0ab33f7550 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.851717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95aba8-5ec2-49a5-9ada-ff0ab33f7550 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.857343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 414/832] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:11 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:11.859142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2b40-7bf9-47a8-b0b5-d5ab06166319 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.859635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2b40-7bf9-47a8-b0b5-d5ab06166319 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.860114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2b40-7bf9-47a8-b0b5-d5ab06166319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.878556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eda96-dcff-4099-864c-49250021b33b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:11.878852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-05bf2b40-7bf9-47a8-b0b5-d5ab06166319 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:11.880300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-05bf2b40-7bf9-47a8-b0b5-d5ab06166319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:11.920463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:11.921472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:11.921956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:11.922445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:11.923511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=075e9c0ef8fd406087d05a6ce36dc4e2) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:11.924439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:11.926685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:11.935476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 419/833] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:11 2022] GET /identity/v3/role_assignments?user.id=075e9c0ef8fd406087d05a6ce36dc4e2&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:11.967066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 415/834] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:11.987490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.102351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a24e8-16f9-4db5-80f3-775cd80f76ff None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:12.110363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:12.140965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:12.141618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.141896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.142169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.153556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:12.159131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:12.160066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:12.160886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:12.165835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 420/835] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:12.167316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf2b40-7bf9-47a8-b0b5-d5ab06166319 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.197259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:12.276541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf2b40-7bf9-47a8-b0b5-d5ab06166319 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:12.312912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:12.313935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.314425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.314969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.315896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:12.316422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:12.318004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:12.327926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 416/836] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:12.338123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.445521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c53662-e622-486f-8277-bd9e33a666d4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:12.483803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73860b1-ba4e-477f-918f-bf1e2f295a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:12.484458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73860b1-ba4e-477f-918f-bf1e2f295a4b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.484740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73860b1-ba4e-477f-918f-bf1e2f295a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.485079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73860b1-ba4e-477f-918f-bf1e2f295a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.496452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73860b1-ba4e-477f-918f-bf1e2f295a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:12.497944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73860b1-ba4e-477f-918f-bf1e2f295a4b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a73860b1-ba4e-477f-918f-bf1e2f295a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:12.513459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:12.514361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:12.515463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:12.515463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:12.515463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:12.515463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:12.515463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 421/837] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:12.523949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.539035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f38a1-c815-4bde-96aa-bdc9937ce493 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.539319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f38a1-c815-4bde-96aa-bdc9937ce493 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.539595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f38a1-c815-4bde-96aa-bdc9937ce493 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.540939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 422/838] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:12.545945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839797b-9066-4468-b679-a1b0a5e81984 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.546443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839797b-9066-4468-b679-a1b0a5e81984 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.546718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839797b-9066-4468-b679-a1b0a5e81984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.565451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c839797b-9066-4468-b679-a1b0a5e81984 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:12.566629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c839797b-9066-4468-b679-a1b0a5e81984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:12.582004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:12.629677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:12.640290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 423/839] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:12.641636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac230-05f1-4a32-bdaf-9aa477b920e2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.641928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac230-05f1-4a32-bdaf-9aa477b920e2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.642264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ac230-05f1-4a32-bdaf-9aa477b920e2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.643734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 424/840] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:12.648941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd9ee2-d1d6-443d-b5b9-1767a4ad3e7c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.649226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd9ee2-d1d6-443d-b5b9-1767a4ad3e7c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.649501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd9ee2-d1d6-443d-b5b9-1767a4ad3e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.653832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac85e4-f22f-49f3-b17a-ef872d56ac46 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:12.667536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-33fd9ee2-d1d6-443d-b5b9-1767a4ad3e7c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:12.668720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-33fd9ee2-d1d6-443d-b5b9-1767a4ad3e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:12.695801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:12.697170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.697447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.697715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.698393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:12.698696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:12.700085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:12.711689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 417/841] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:12.712487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.751059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 425/842] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:12.751059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd9ee2-d1d6-443d-b5b9-1767a4ad3e7c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.826012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a85f4-d627-4a14-a5ad-9123a530bc70 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:12.837435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:12.865239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:12.865956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd9ee2-d1d6-443d-b5b9-1767a4ad3e7c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:12.866427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.866707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.867007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.867734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:12.869204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:12.878399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:12.879378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:12.880194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 418/843] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:12.885933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.901766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:12.903703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:12.904434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:12.904785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:12.905127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:12.912172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:12.914185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:12.926799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:12.927812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:12.929297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 426/844] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:12.929864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:12.996491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0d1d-7822-4266-8720-afe77d9d0f34 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.030217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.030853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.031156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.031425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.031589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa9f08-ebd5-415f-bc76-c906e94ccf77 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.042615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:13.047520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:13.048640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 419/845] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:13.049497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.072624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.073485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.073772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.074046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.074719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:13.075069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.076412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.080981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 427/846] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:13.081741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.084409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.136534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.169000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddf1e4-e9cb-4678-b5a4-9f0dce80c1c0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.191210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb868-a97b-4e23-a803-6fdfe0982f72 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.203472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.204094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.204477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.204656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.205436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:13.205745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.207191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.229765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 420/847] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:13.229765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.230007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.230007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.230007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.230007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.230295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:13.230295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.230748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.238806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 428/848] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:13.239621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.305250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.338338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88aa38-8132-4dc8-8be7-e497ee0e361f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.344702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50426e8b-c58d-4664-8696-4b81d7bc5bab None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.368070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.375061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.375676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.375959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.376232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.376905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:13.377208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.378565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.380455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.381477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.381734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.381943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.395276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 421/849] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:13.395276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.457840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.459874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.506243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d04f92-fa2c-4409-b752-a775b9692948 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.528121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.545414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.546377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.546845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.547246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.559937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.561663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.573078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 429/850] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:13.578955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:13.584741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:13.585631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:13.586614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:13.586614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:13.586614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:13.586614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:13.590718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 422/851] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:13.591655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.648543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.712469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aff27-ec91-4dac-a67a-7948858fc5c4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.722058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e233968-f8f4-4d61-a301-b31e6b4bfb00 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.755174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.755797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.756610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.756912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.757688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:13.758057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.759457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.766561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.767930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.768216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.768494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.777800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 423/852] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:13.778832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.780916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.782366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:13.803556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:13.804544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:13.805446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:13.805446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:13.805446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:13.805446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:13.806362 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 430/853] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:13.810408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.835728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.901286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df8457-11a6-44bc-b2f3-a4fcbca76d9f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.911534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:13.932554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eaebd-e4ce-4e19-83a1-868704f0a972 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:13.948062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.948782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.949069 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.949347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.956878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.958327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.969995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:13.970795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:13.971118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:13.971984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:13.972932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:13.973448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:13.973448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:13.973448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:13.973448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:13.973689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 424/854] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:13.973763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:13.983957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:13.986795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 431/855] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:13.992685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.088714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.104664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2798c4e-d366-4dff-ba4a-2c5698ea1f03 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.116347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f07c5-db47-42ee-b9eb-9c521dc6251d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.140328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb3ffb-21ac-4360-a9be-636986d7dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.140954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb3ffb-21ac-4360-a9be-636986d7dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.141234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb3ffb-21ac-4360-a9be-636986d7dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.141682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb3ffb-21ac-4360-a9be-636986d7dbe1 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.142317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb3ffb-21ac-4360-a9be-636986d7dbe1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:14.142594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb3ffb-21ac-4360-a9be-636986d7dbe1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.144037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb3ffb-21ac-4360-a9be-636986d7dbe1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.151968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fa07e-9e39-4f11-9aeb-5c247f861368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.152605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fa07e-9e39-4f11-9aeb-5c247f861368 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.152877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fa07e-9e39-4f11-9aeb-5c247f861368 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.153148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fa07e-9e39-4f11-9aeb-5c247f861368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.160092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fa07e-9e39-4f11-9aeb-5c247f861368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.161460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fa07e-9e39-4f11-9aeb-5c247f861368 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.162473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 425/856] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:14.163821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c246bd7-10fe-4052-8fd1-10e9c942cedd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.164114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c246bd7-10fe-4052-8fd1-10e9c942cedd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.164389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c246bd7-10fe-4052-8fd1-10e9c942cedd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.169685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 426/857] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:14.170518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c246bd7-10fe-4052-8fd1-10e9c942cedd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a10fa07e-9e39-4f11-9aeb-5c247f861368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:14.177156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:14.178695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:14.179823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:14.182219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 432/858] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:14.184211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1849d3b-f4f2-4409-b4c5-1e6c2438558e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.184655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1849d3b-f4f2-4409-b4c5-1e6c2438558e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.185318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1849d3b-f4f2-4409-b4c5-1e6c2438558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.197613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.207502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b1849d3b-f4f2-4409-b4c5-1e6c2438558e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:14.208701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b1849d3b-f4f2-4409-b4c5-1e6c2438558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:14.275775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 433/859] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:14.276603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1849d3b-f4f2-4409-b4c5-1e6c2438558e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.289064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c246bd7-10fe-4052-8fd1-10e9c942cedd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.326153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.327087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.327366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.327632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.350693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.388006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1849d3b-f4f2-4409-b4c5-1e6c2438558e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.396561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.398036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.423169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.423874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.424051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.424264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.425154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.425900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:14.426198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.427639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.439463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 434/860] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:14.440262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.510052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 427/861] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:14.511199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.551006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb1d17-d7c6-40ca-b24b-c905b60357e9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.580298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.592088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.593213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.593820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.594165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.594807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.596321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:14.606554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:14.607518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:14.608354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 435/862] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:14.609224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.624387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981926ad-e265-4d94-9519-72b7dab6ed3a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.650616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.663165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.663785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.664064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.664339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.665117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=add21760f9ed4b82b942c5dd9a6ed096) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:14.665422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.666852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.672599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 428/863] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/role_assignments?user.id=add21760f9ed4b82b942c5dd9a6ed096&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:14.673395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.734013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61ccb3-c2a9-42ab-8283-1d60eb5f246c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.774329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.775429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.775894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.776379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.791410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.791929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf71271-3982-459a-aa1a-ee921c035e47 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.793814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:14.818983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:14.820019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:14.820912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:14.820912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:14.820912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:14.820912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:14.820912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 436/864] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:14.825664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.840485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:14.842055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:14.842340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:14.842615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:14.843815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:14.844174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:14.848323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:14.855611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 429/865] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:14.864863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:14.891678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.932568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:14.969361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0a01e-cb94-40d5-8b24-a171baa1d627 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:14.986978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812a826-e979-422d-9ce9-06facc5b719d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:15.022443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:15.023637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.024048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.024335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.025282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:15.025595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:15.027802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:15.031156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:15.031911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.032295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.032675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.033445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:15.033853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:15.035409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:15.041521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 430/866] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:15.042673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 437/867] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:15.048693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:15.169096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab337-491e-4cc4-a19c-20361893f5d5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:15.187619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:15.208346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:15.208990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.209825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.210106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.217080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:15.218553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:15.229986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:15.230936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:15.231832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:15.234942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 431/868] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:15.237242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:15.345678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220785-ad98-4915-a67f-4b41045d6eb9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:15.347558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:15.384443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a8633-b0c4-48cc-8a58-bd4b550215b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:15.385072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a8633-b0c4-48cc-8a58-bd4b550215b3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.385354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a8633-b0c4-48cc-8a58-bd4b550215b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.385630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a8633-b0c4-48cc-8a58-bd4b550215b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.386374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a8633-b0c4-48cc-8a58-bd4b550215b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:15.386682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a8633-b0c4-48cc-8a58-bd4b550215b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:15.388135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a8633-b0c4-48cc-8a58-bd4b550215b3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:15.400803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 438/869] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:15.406562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:15.504116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4b264-2a14-4b37-80f9-9ba9af59bfa3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.504309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4b264-2a14-4b37-80f9-9ba9af59bfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.504580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4b264-2a14-4b37-80f9-9ba9af59bfa3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.506695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 439/870] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:15.511701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71d4f2-683e-4df3-8c14-e375d78b73a6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.511990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71d4f2-683e-4df3-8c14-e375d78b73a6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.512112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89269b6-729e-4c99-bd49-cc86044c8eef None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:15.512285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71d4f2-683e-4df3-8c14-e375d78b73a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.534461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2f71d4f2-683e-4df3-8c14-e375d78b73a6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:15.535680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2f71d4f2-683e-4df3-8c14-e375d78b73a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:15.548961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:15.550170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.550655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.551160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.552364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=5f26e831b05740c9a1d510f107e74b57) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:15.552749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:15.554803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:15.561745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 432/871] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:15 2022] GET /identity/v3/role_assignments?user.id=5f26e831b05740c9a1d510f107e74b57&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:15.613336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 440/872] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:15.636601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:15.692756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:15.758830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a96ecd-1259-45ba-844f-2e8723af27a1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:15.792265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0899c-92aa-4e7c-8c90-3d82dacdc99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:15.792883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0899c-92aa-4e7c-8c90-3d82dacdc99c None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:15.793166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0899c-92aa-4e7c-8c90-3d82dacdc99c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:15.793439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0899c-92aa-4e7c-8c90-3d82dacdc99c None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:15.794136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0899c-92aa-4e7c-8c90-3d82dacdc99c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:15.795530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0899c-92aa-4e7c-8c90-3d82dacdc99c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:15.817386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 433/873] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:15 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:16.870630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba71125-54af-4b95-be9b-dc59823a6c4b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:16.871207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba71125-54af-4b95-be9b-dc59823a6c4b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:16.871207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba71125-54af-4b95-be9b-dc59823a6c4b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:16.872839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 441/874] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:16.878299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695eddb-e25d-4e1a-9c68-ff68fbf6eb34 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:16.878713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695eddb-e25d-4e1a-9c68-ff68fbf6eb34 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:16.879034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695eddb-e25d-4e1a-9c68-ff68fbf6eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:16.898092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a695eddb-e25d-4e1a-9c68-ff68fbf6eb34 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:16.899308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a695eddb-e25d-4e1a-9c68-ff68fbf6eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:16.983057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 434/875] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:16.998004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba71125-54af-4b95-be9b-dc59823a6c4b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:17.093072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:17.103451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba71125-54af-4b95-be9b-dc59823a6c4b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:17.139994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8208f-64f9-4356-867b-eb456f57f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:17.141005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8208f-64f9-4356-867b-eb456f57f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.141505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8208f-64f9-4356-867b-eb456f57f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.142012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8208f-64f9-4356-867b-eb456f57f3a5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.142834 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8208f-64f9-4356-867b-eb456f57f3a5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:17.144460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8208f-64f9-4356-867b-eb456f57f3a5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-baf8208f-64f9-4356-867b-eb456f57f3a5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:17.153914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:17.154943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:17.155799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:17.158528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 442/876] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:17.164335 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695eddb-e25d-4e1a-9c68-ff68fbf6eb34 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:17.191317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dca18-724a-4c00-8f9f-5843a3c9a82f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.191921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dca18-724a-4c00-8f9f-5843a3c9a82f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.192273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dca18-724a-4c00-8f9f-5843a3c9a82f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.192375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:17.194501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 443/877] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:17.209073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58b1f-206f-4ea1-9c67-d3224ab2fc28 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.209439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58b1f-206f-4ea1-9c67-d3224ab2fc28 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.209802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58b1f-206f-4ea1-9c67-d3224ab2fc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.233880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4bb58b1f-206f-4ea1-9c67-d3224ab2fc28 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:17.235438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4bb58b1f-206f-4ea1-9c67-d3224ab2fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:17.271802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695eddb-e25d-4e1a-9c68-ff68fbf6eb34 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:17.316917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:17.317691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.317972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.318246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.319587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:17.319895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:17.321985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:17.326158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 444/878] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:17.335074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 435/879] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:17.336338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58b1f-206f-4ea1-9c67-d3224ab2fc28 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:17.357678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:17.427057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:17.444267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58b1f-206f-4ea1-9c67-d3224ab2fc28 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:17.480511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:17.486010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a8b5b-c915-43b3-8ebb-c434d3aa9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:17.486962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8b5b-c915-43b3-8ebb-c434d3aa9c45 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.487526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8b5b-c915-43b3-8ebb-c434d3aa9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.488140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8b5b-c915-43b3-8ebb-c434d3aa9c45 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.489068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8b5b-c915-43b3-8ebb-c434d3aa9c45 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:17.489647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8b5b-c915-43b3-8ebb-c434d3aa9c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:17.491426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077b16c-dab7-41fa-90e2-eafebf656613 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:17.492003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8b5b-c915-43b3-8ebb-c434d3aa9c45 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:17.503376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 445/880] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:17.528970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333efc3-8fc2-4616-a93e-e8c94560b8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:17.529614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333efc3-8fc2-4616-a93e-e8c94560b8d8 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.529893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333efc3-8fc2-4616-a93e-e8c94560b8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.530168 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333efc3-8fc2-4616-a93e-e8c94560b8d8 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.530896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333efc3-8fc2-4616-a93e-e8c94560b8d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:17.532327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333efc3-8fc2-4616-a93e-e8c94560b8d8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:17.555784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 436/881] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:17 2022] POST /identity/v3/services => generated 269 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:18:17.733881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cb747-2d6b-41dd-9f86-0890d6a95784 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.734208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cb747-2d6b-41dd-9f86-0890d6a95784 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.734431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cb747-2d6b-41dd-9f86-0890d6a95784 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.736046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 446/882] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:17.741776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5739a5f-6b9c-4fad-89ad-1528a788bb6f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.742062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5739a5f-6b9c-4fad-89ad-1528a788bb6f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.742336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5739a5f-6b9c-4fad-89ad-1528a788bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.761441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a5739a5f-6b9c-4fad-89ad-1528a788bb6f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:17.762630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a5739a5f-6b9c-4fad-89ad-1528a788bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:17.836562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 437/883] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:17.860908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cb747-2d6b-41dd-9f86-0890d6a95784 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:17.920275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0151b-d4e1-4445-8c29-eb547efc5a1d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.921291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0151b-d4e1-4445-8c29-eb547efc5a1d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.921765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0151b-d4e1-4445-8c29-eb547efc5a1d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.923576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 438/884] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:17.929674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a3c04-4034-4cf4-ba94-b1073e193192 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:17.929796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a3c04-4034-4cf4-ba94-b1073e193192 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:17.930072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a3c04-4034-4cf4-ba94-b1073e193192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:17.949339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-195a3c04-4034-4cf4-ba94-b1073e193192 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:17.950248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-195a3c04-4034-4cf4-ba94-b1073e193192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:17.977079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cb747-2d6b-41dd-9f86-0890d6a95784 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.015442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.016012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.016293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.016569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.023765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 439/885] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:18.034014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:18.039925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:18.040859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:18.041687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:18.044254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 447/886] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Aug 6 14:18:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:18.048646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a3c04-4034-4cf4-ba94-b1073e193192 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.050575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.058724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.067697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.167061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1298a2-f30a-486a-a505-8aab4c034a4f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.181741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a3c04-4034-4cf4-ba94-b1073e193192 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.203943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.204811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.205311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.205763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.206642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:18.207158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.208696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.217979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 448/887] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:18.223601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.229964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.230616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.230921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.231191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.231820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.233182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:18.245401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:18.246383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:18.247250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:18.251018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 440/888] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:18.252699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.309753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.343482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53573052-f573-45cf-9c4a-8788408a45b0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.353611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.380554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.381728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.382010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.382316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.394540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.396118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.396410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a215f7d-f80f-446d-856c-4abdf520ea23 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:18.411841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:18.412931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:18.413955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:18.413955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:18.413955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:18.413955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:18.419588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 449/889] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:18.420237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.445238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.446329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.446832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.447552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.448809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:18.449522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.451708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.463635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 441/890] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:18.465064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.535009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e1144-ba6d-4039-bc03-ba2b1569bcb0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.561308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.577327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.577965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.578283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.578562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.579575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:18.579886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.581250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.603719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68459e-c39b-4031-a5a5-5dd14339ff1f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.603941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 450/891] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:18.604759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.646516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.647653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.647996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.648339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.649085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:18.649453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.651020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.653335 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.656719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 442/892] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:18.663215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.729461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfcd98-4f58-4bbc-94fd-da77c9db64e5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.770290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.770972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.771353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.771678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.778096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781a444-3ac1-4e3a-9052-8579e5debdba None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.779837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.781503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:18.794115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:18.795108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:18.796045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 451/893] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:18.801326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.819359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.819915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.820194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.820464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.821206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] RBAC: Adding query filter params (interface=public, service_id=5b6a60b3f8e74b3cb3a9d074ce57d450, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:18.821518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.822991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.832865 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 443/894] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/endpoints?service_id=5b6a60b3f8e74b3cb3a9d074ce57d450&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:18.833841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.852363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.900885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:18.914177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ae3a5-5352-49de-81e5-519704368d21 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.944307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df98ab-c87c-4386-b09a-bcbf562f1941 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:18.951032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.951778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.952163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.952541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.953629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:18.954053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.955703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:18.985501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 452/895] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:18.986710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:18.988309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:18.988931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:18.989208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:18.989479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:18.990155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:18.992135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:19.005275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 444/896] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:18:19.114675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d11ad-9e57-4c99-8d14-27fbf566e6cf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:19.123718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:19.153517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:19.154147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.154429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.154705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.228187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:19.229637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:19.332273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 453/897] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:18 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:19.338375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:19.443694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce178e-9b4c-49f2-8900-0a6abd5a2e3a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:19.470928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:19.481896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f9459-65ff-4838-8d70-9450da6e6241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:19.482892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f9459-65ff-4838-8d70-9450da6e6241 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.483374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f9459-65ff-4838-8d70-9450da6e6241 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.483857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f9459-65ff-4838-8d70-9450da6e6241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.495453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f9459-65ff-4838-8d70-9450da6e6241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:19.497182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f9459-65ff-4838-8d70-9450da6e6241 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-aa4f9459-65ff-4838-8d70-9450da6e6241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:19.511748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:19.512640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:19.513506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:19.513506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:19.513506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:19.513506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:19.515663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 445/898] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:19.521388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:19.524533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:19.626978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb5628-85da-4905-8e68-51783b318945 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:19.631290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04dbb4-7e84-453f-9024-68cba738ab3e None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.631579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04dbb4-7e84-453f-9024-68cba738ab3e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.631855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04dbb4-7e84-453f-9024-68cba738ab3e None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.633340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 446/899] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:19.638665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59141a8-e63f-447f-b9a9-106926e20cd7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.639237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59141a8-e63f-447f-b9a9-106926e20cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.639769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59141a8-e63f-447f-b9a9-106926e20cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.661074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f59141a8-e63f-447f-b9a9-106926e20cd7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:19.664068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f59141a8-e63f-447f-b9a9-106926e20cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:19.668009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225e83-03ec-4978-a51c-762fe4271918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:19.668641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225e83-03ec-4978-a51c-762fe4271918 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.668921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225e83-03ec-4978-a51c-762fe4271918 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.669190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225e83-03ec-4978-a51c-762fe4271918 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.669971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42225e83-03ec-4978-a51c-762fe4271918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:19.670277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42225e83-03ec-4978-a51c-762fe4271918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:19.671650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42225e83-03ec-4978-a51c-762fe4271918 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:19.683647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 454/900] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:19.685098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97789f7-1c09-4edb-9b46-f90bebe9be20 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.685593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97789f7-1c09-4edb-9b46-f90bebe9be20 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.686065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97789f7-1c09-4edb-9b46-f90bebe9be20 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.691100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 455/901] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:19.692280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97789f7-1c09-4edb-9b46-f90bebe9be20 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:19.739963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 447/902] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:19.741251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606059ca-bdfe-4ef1-8111-cc07cc4d6f70 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.741542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606059ca-bdfe-4ef1-8111-cc07cc4d6f70 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.741817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606059ca-bdfe-4ef1-8111-cc07cc4d6f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.760160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-606059ca-bdfe-4ef1-8111-cc07cc4d6f70 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:19.761781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-606059ca-bdfe-4ef1-8111-cc07cc4d6f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:19.764751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:19.808029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97789f7-1c09-4edb-9b46-f90bebe9be20 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:19.836338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 448/903] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:19.839713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606059ca-bdfe-4ef1-8111-cc07cc4d6f70 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:19.852715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:19.853497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.853882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.854261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.862204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:19.863786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:19.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:19.876330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:19.877252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 456/904] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:19.877784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:19.895691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:19.960022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606059ca-bdfe-4ef1-8111-cc07cc4d6f70 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:19.992232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27c0d3-7bc1-43a3-a74c-c24cad769c84 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:19.995536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:19.996241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:19.997611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:19.997611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:19.997611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:19.999912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.016323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 449/905] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:19 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:20.025974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:20.027936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:20.056411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:20.057281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:20.057640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:20.057985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:20.059434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:20.061221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:20.075064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:20.076085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:20.076969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:20.081643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 457/906] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Aug 6 14:18:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:20.082914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:20.157927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:20.165341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af6eda-8073-4a15-8c78-c72aebac8f49 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:20.210641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:20.211342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:20.211623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:20.211899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:20.212668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:20.212978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:20.213076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca643b-2f93-4d0b-8b27-deb087977830 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:20.214448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.229127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 450/907] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:20.234857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:20.262121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:20.263377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:20.264329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:20.265179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:20.266321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:20.267252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:20.269431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.277062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 458/908] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:20.282992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:20.355250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:20.357861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1274c-c66d-4c7d-b29c-b5979acf45c7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:20.407044 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:20.408296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:20.409052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:20.409913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:20.411204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] RBAC: Adding query filter params (role.id=7e2a69597b9f47a598002ebca7e97820, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=5f26e831b05740c9a1d510f107e74b57) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:20.411723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:20.413066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe3ba9-7451-4e44-a10b-20e9b083de81 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:20.414265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.422203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 451/909] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:20 2022] GET /identity/v3/role_assignments?user.id=5f26e831b05740c9a1d510f107e74b57&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=7e2a69597b9f47a598002ebca7e97820 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:20.456196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:20.456979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:20.457331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:20.457677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:20.458510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:20.459000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:20.460528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.467116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 459/910] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:20.472680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:20.505037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:20.593069 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e39fc5-143f-4d29-be54-1c6e5e150787 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:20.632096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd828f88-2e06-4ad1-a963-56782c9b30fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:20.632722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd828f88-2e06-4ad1-a963-56782c9b30fa None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:20.632998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd828f88-2e06-4ad1-a963-56782c9b30fa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:20.633277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd828f88-2e06-4ad1-a963-56782c9b30fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:20.634029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd828f88-2e06-4ad1-a963-56782c9b30fa None admin] RBAC: Adding query filter params (interface=public, service_id=b0efe5f962cb443fa351ab046754daa5, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:20.634340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd828f88-2e06-4ad1-a963-56782c9b30fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:20.635756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd828f88-2e06-4ad1-a963-56782c9b30fa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.646134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 452/911] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:18:20 2022] GET /identity/v3/endpoints?service_id=b0efe5f962cb443fa351ab046754daa5&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:20.647518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:20.758847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56031fa-29fa-4cf4-97fc-33ec3569a19f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:20.770703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:20.798450 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d03270-247e-47e2-84c4-9a31117986ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:20.799268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d03270-247e-47e2-84c4-9a31117986ef None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:20.799654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d03270-247e-47e2-84c4-9a31117986ef None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:20.800282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d03270-247e-47e2-84c4-9a31117986ef None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:20.801040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d03270-247e-47e2-84c4-9a31117986ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:20.802501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d03270-247e-47e2-84c4-9a31117986ef None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:20.809430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 460/912] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:20 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:18:21.192099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc61a89-0a89-44f7-b42a-3aeab7f6521b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:21.192624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc61a89-0a89-44f7-b42a-3aeab7f6521b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:21.192699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc61a89-0a89-44f7-b42a-3aeab7f6521b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:21.194813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 453/913] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:21.200184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248434-3b57-4c9d-9fe8-810faa53df6a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:21.200620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248434-3b57-4c9d-9fe8-810faa53df6a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:21.200798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248434-3b57-4c9d-9fe8-810faa53df6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:21.219660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-37248434-3b57-4c9d-9fe8-810faa53df6a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:21.220855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-37248434-3b57-4c9d-9fe8-810faa53df6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:21.291675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 461/914] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:21.314168 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc61a89-0a89-44f7-b42a-3aeab7f6521b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:21.411072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:21.429119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc61a89-0a89-44f7-b42a-3aeab7f6521b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:21.465872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:21.466600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:21.466896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:21.467288 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:21.467922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:21.469539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:21.480294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:21.481263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:21.482103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 454/915] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:21.489364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37248434-3b57-4c9d-9fe8-810faa53df6a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:21.599164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37248434-3b57-4c9d-9fe8-810faa53df6a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:21.601590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:21.641737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:21.644054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:21.644382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:21.644633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:21.645885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:21.646475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:21.648468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:21.654073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 462/916] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:21.663931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:21.772039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436383ac-6f78-45b7-8760-7da52d36687e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:21.807853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475ed63-14e8-428d-a067-72ff9a233b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:21.808477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ed63-14e8-428d-a067-72ff9a233b8e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:21.808759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ed63-14e8-428d-a067-72ff9a233b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:21.809034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ed63-14e8-428d-a067-72ff9a233b8e None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:21.809716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475ed63-14e8-428d-a067-72ff9a233b8e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:21.810024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475ed63-14e8-428d-a067-72ff9a233b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:21.811430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475ed63-14e8-428d-a067-72ff9a233b8e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:21.816786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 455/917] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Aug 6 14:18:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:21.822936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:21.931002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7428-ac78-450e-a244-aa4f7cf4a424 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:21.968912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c56091-7b16-4900-a338-1522886d9f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:21.969544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c56091-7b16-4900-a338-1522886d9f4f None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:21.969831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c56091-7b16-4900-a338-1522886d9f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:21.970107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c56091-7b16-4900-a338-1522886d9f4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:21.971033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c56091-7b16-4900-a338-1522886d9f4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:21.972444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c56091-7b16-4900-a338-1522886d9f4f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:21.995427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 463/918] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:22.077454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9582-05b2-4e38-9113-585ae9278842 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.077665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9582-05b2-4e38-9113-585ae9278842 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.077929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9582-05b2-4e38-9113-585ae9278842 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.079890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 456/919] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:22.086704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da262a7f-caa8-42e2-9580-d8136b542909 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.087013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da262a7f-caa8-42e2-9580-d8136b542909 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.087302 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da262a7f-caa8-42e2-9580-d8136b542909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.107735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-da262a7f-caa8-42e2-9580-d8136b542909 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:22.109304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-da262a7f-caa8-42e2-9580-d8136b542909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:22.131221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:22.200488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 464/920] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:22.223462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca9582-05b2-4e38-9113-585ae9278842 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:22.242383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:22.336503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca9582-05b2-4e38-9113-585ae9278842 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:22.373240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:22.374260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.374844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.375366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.376194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:22.377761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:22.387396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:22.388453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:22.389293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:22.392158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 457/921] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:22.401588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da262a7f-caa8-42e2-9580-d8136b542909 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:22.503339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da262a7f-caa8-42e2-9580-d8136b542909 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:22.540755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f577aa-6af7-4fe1-aabb-e85f3bae8b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:22.541372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f577aa-6af7-4fe1-aabb-e85f3bae8b2d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.541650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f577aa-6af7-4fe1-aabb-e85f3bae8b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.541921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f577aa-6af7-4fe1-aabb-e85f3bae8b2d None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.542591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f577aa-6af7-4fe1-aabb-e85f3bae8b2d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:22.542906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f577aa-6af7-4fe1-aabb-e85f3bae8b2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:22.544255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f577aa-6af7-4fe1-aabb-e85f3bae8b2d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:22.549495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 465/922] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:22.555373 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:22.628505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89b77c-8f3f-4e72-9ff8-ef397a2fad11 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.628778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89b77c-8f3f-4e72-9ff8-ef397a2fad11 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.629012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89b77c-8f3f-4e72-9ff8-ef397a2fad11 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.630418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 466/923] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:22.635215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3e1d0-b711-4808-84e3-a4f05fdffa4b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.635496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3e1d0-b711-4808-84e3-a4f05fdffa4b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.635768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3e1d0-b711-4808-84e3-a4f05fdffa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.654369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f2c3e1d0-b711-4808-84e3-a4f05fdffa4b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:22.655713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f2c3e1d0-b711-4808-84e3-a4f05fdffa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:22.666176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:22.671241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:22.679477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c6995-88d5-42d4-9483-63848a3e01dd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:22.716869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:22.717491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.717848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.718039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.720067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:22.720383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:22.721827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:22.731786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 458/924] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:22.733049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:22.743417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 467/925] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:22.767959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3e1d0-b711-4808-84e3-a4f05fdffa4b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:22.845365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ab58c-eb10-4415-8d95-884eb023ed39 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:22.884172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a710e4-e527-49b5-af97-e883fcf07f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:22.884792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a710e4-e527-49b5-af97-e883fcf07f1d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.885072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a710e4-e527-49b5-af97-e883fcf07f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.885346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a710e4-e527-49b5-af97-e883fcf07f1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.886089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a710e4-e527-49b5-af97-e883fcf07f1d None admin] RBAC: Adding query filter params (interface=public, service_id=7ea5db7078494d5e9d2d8013d5cbc770, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:22.886397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a710e4-e527-49b5-af97-e883fcf07f1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:22.887787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a710e4-e527-49b5-af97-e883fcf07f1d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:22.892418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3e1d0-b711-4808-84e3-a4f05fdffa4b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:22.898068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 459/926] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:18:22 2022] GET /identity/v3/endpoints?service_id=7ea5db7078494d5e9d2d8013d5cbc770&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:22.898068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1660b-ab14-4602-aef8-0f9df4856033 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.898068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1660b-ab14-4602-aef8-0f9df4856033 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.898068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1660b-ab14-4602-aef8-0f9df4856033 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.898068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 460/927] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:22.898740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1660b-ab14-4602-aef8-0f9df4856033 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:22.908231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:22.946679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd8ca-d223-4837-a640-f199bf74afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:22.947655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd8ca-d223-4837-a640-f199bf74afc2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.948089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd8ca-d223-4837-a640-f199bf74afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.948535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd8ca-d223-4837-a640-f199bf74afc2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.969282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fd8ca-d223-4837-a640-f199bf74afc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-679fd8ca-d223-4837-a640-f199bf74afc2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:22.974364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:22.975471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:22.976425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:22.983373 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 468/928] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Aug 6 14:18:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:22.984910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f625f-8053-4170-9b38-3617b0e35744 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:22.985194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f625f-8053-4170-9b38-3617b0e35744 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:22.985469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f625f-8053-4170-9b38-3617b0e35744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:22.994419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:23.007024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a42f625f-8053-4170-9b38-3617b0e35744 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:23.008249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a42f625f-8053-4170-9b38-3617b0e35744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:23.009106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1660b-ab14-4602-aef8-0f9df4856033 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.047907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.048881 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.049158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.049428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.050329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.051795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.066909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 461/929] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:22 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:18:23.066909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.099066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 469/930] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:23.115634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f625f-8053-4170-9b38-3617b0e35744 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.159906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:23.196882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ddd8a-6684-449e-8ae7-9e3aeaa634b9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.235076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.235872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.236170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.236453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.237283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:23.237593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.239128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.242493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f625f-8053-4170-9b38-3617b0e35744 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.254479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 462/931] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:23.255404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.288879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.289534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.289850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.290091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.290751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.292177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.293977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:23.306757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:23.307925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:23.309065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 470/932] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:23.314857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.382936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791372d2-c92d-4de5-bc95-0a86c0ee2dff None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.416436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:23.423685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.424470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.425127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.425497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.434535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0855c-500f-4e61-85f3-a2139924b7b8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.445273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.446954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:23.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:23.463578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:23.464672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:23.464672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:23.464672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:23.464672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:23.466357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 463/933] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:23.472931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.477861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.478672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.479224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.479770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.480521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:23.480831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.482263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.492986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 471/934] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:23.494146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.561098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:23.613701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1049eb3-fd3a-4e09-8124-6e6f8874f487 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.614160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70fd3f-0682-40fc-93b3-8f4dccafcc76 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.655935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.656574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.656854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.657127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.657828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:23.658134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.659703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.664169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.665037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.665517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.666001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.666956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:23.667480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.669026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.679120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 472/935] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:23.680349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 464/936] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:23.681435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.684114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:23.734428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:23.808635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5f15-bb90-43ab-a78d-c67e5bb6e6b1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.809270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e45383-a3bf-4277-82f8-3ed202f504a6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:23.836953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:23.856746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.857660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.858153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.858577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.858693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:23.859536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:23.859742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.860009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:23.860379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:23.861773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.868286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:23.869872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:23.884366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:23.885587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:23.886737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:23.893330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 473/937] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:23.895008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 465/938] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:23.900050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.048002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b356a1e-b8ef-4bbd-a3bd-aef714fee357 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:24.091747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd742d7-bb1d-4dbe-8fe4-8e06823508d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:24.092636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd742d7-bb1d-4dbe-8fe4-8e06823508d8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.093096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd742d7-bb1d-4dbe-8fe4-8e06823508d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.093519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd742d7-bb1d-4dbe-8fe4-8e06823508d8 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.094487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd742d7-bb1d-4dbe-8fe4-8e06823508d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:24.094901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd742d7-bb1d-4dbe-8fe4-8e06823508d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:24.096334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd742d7-bb1d-4dbe-8fe4-8e06823508d8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:24.115975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 474/939] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Aug 6 14:18:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:24.121695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.228519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:24.232735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e814365-c233-488b-97d3-d60750f8244b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:24.263788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d7d3c-23b3-4fcf-9fa1-9c76dbdae272 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.264294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d7d3c-23b3-4fcf-9fa1-9c76dbdae272 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.264406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d7d3c-23b3-4fcf-9fa1-9c76dbdae272 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.265923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 475/940] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:24.270980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6a27d-c987-4bf3-a82b-a2c4638e3b31 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.271115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:24.272090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6a27d-c987-4bf3-a82b-a2c4638e3b31 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.272090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6a27d-c987-4bf3-a82b-a2c4638e3b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.272225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.272474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.272752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.285230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:24.287202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:24.289957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6ae6a27d-c987-4bf3-a82b-a2c4638e3b31 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:24.291191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6ae6a27d-c987-4bf3-a82b-a2c4638e3b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:24.306794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:24.307885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:24.308811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:24.308811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:24.308811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:24.308811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:24.311465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 466/941] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:24.317234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.332232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:24.368945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 476/942] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:24.392551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6a27d-c987-4bf3-a82b-a2c4638e3b31 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.425310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3880e-6207-4ddf-9ff3-4687a61069a5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:24.460829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:24.461560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.461915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.462272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.463090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:24.463480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:24.464927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:24.482337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 467/943] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:24.484303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.500056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6a27d-c987-4bf3-a82b-a2c4638e3b31 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:24.500594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:24.537134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:24.537752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.538035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.538306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.539000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:24.540530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:24.543117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING keystone.server.flask.common [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 06 14:18:24.561859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.sql.core [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 14:18:24.561859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 14:18:24.561859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [parameters: {'id': '3efa9948c93a4ec987c3e512e0d0e867', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 14:18:24.561859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80151) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3efa9948c93a4ec987c3e512e0d0e867.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3efa9948c93a4ec987c3e512e0d0e867. Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 14:18:24.584600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 06 14:18:24.585468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application yield resource Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise self.value Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application session.commit() Aug 06 14:18:24.586407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.flush() Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application compat.raise_( Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 14:18:24.587349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 14:18:24.588499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 14:18:24.589285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [parameters: {'id': '3efa9948c93a4ec987c3e512e0d0e867', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:24.590134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:24.590938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3efa9948c93a4ec987c3e512e0d0e867. Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.592229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 477/944] 10.222.0.17 () {60 vars in 1140 bytes} [Sat Aug 6 14:18:24 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:24.596046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.603544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d9f37-e051-4b4a-a7c9-d68a1578e925 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:24.626682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:24.654702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:24.655879 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.656595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.657071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.666808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:24.669041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:24.686043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:24.687200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:24.688251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:24.695308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 468/945] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:24.696366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.729209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b3c1f-042f-4f82-b9ca-ae452aca7c84 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:24.784154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:24.787828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:24.788735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.789209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.789694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.803634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:24.805638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:24.819177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7b32-f849-4e55-92ce-0ec98ccccdeb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:24.823733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:24.824779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:24.825850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:24.827944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 478/946] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:24.833326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.874061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:24.874775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:24.875165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:24.875517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:24.876349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:24.876701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:24.878375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:24.898415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 469/947] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:24.899564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:24.968315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:24.991058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b86b2-0150-43b4-8de7-12d529292f2a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:25.020519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be0a37-76fb-4e21-ba9e-9d75c6b6341a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:25.027920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e1e79-5d67-4276-a87d-63b9b97de831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:25.028664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e1e79-5d67-4276-a87d-63b9b97de831 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.029050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e1e79-5d67-4276-a87d-63b9b97de831 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.029437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e1e79-5d67-4276-a87d-63b9b97de831 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.030275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e1e79-5d67-4276-a87d-63b9b97de831 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:25.030687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e1e79-5d67-4276-a87d-63b9b97de831 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:25.032253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e1e79-5d67-4276-a87d-63b9b97de831 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:25.058001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:25.060902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 479/948] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:25.065476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aca81-3636-477e-8144-8d971ae498ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:25.066523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aca81-3636-477e-8144-8d971ae498ef None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.067033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aca81-3636-477e-8144-8d971ae498ef None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.067786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aca81-3636-477e-8144-8d971ae498ef None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/5f26e831b05740c9a1d510f107e74b57/roles/7e2a69597b9f47a598002ebca7e97820` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.071163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aca81-3636-477e-8144-8d971ae498ef None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=5f26e831b05740c9a1d510f107e74b57, role_id=7e2a69597b9f47a598002ebca7e97820)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:25.072975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aca81-3636-477e-8144-8d971ae498ef None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:25.091336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 470/949] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:18:24 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/5f26e831b05740c9a1d510f107e74b57/roles/7e2a69597b9f47a598002ebca7e97820 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:18:25.315232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d8d57-5851-4564-b086-11ac9f6b6743 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.315769 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d8d57-5851-4564-b086-11ac9f6b6743 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.315769 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d8d57-5851-4564-b086-11ac9f6b6743 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.317405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 480/950] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:25.322843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11414b06-5f9a-4e29-ba71-65f9b2a2ff52 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.323301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11414b06-5f9a-4e29-ba71-65f9b2a2ff52 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.323403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11414b06-5f9a-4e29-ba71-65f9b2a2ff52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.342147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-11414b06-5f9a-4e29-ba71-65f9b2a2ff52 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:25.343363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-11414b06-5f9a-4e29-ba71-65f9b2a2ff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:25.412627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 471/951] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:25.435064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d8d57-5851-4564-b086-11ac9f6b6743 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:25.547233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d8d57-5851-4564-b086-11ac9f6b6743 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:25.583034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:25.584527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.584953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.585431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.586211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:25.587736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:25.604835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:25.606058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:25.607005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:25.609401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 481/952] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:25.616435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11414b06-5f9a-4e29-ba71-65f9b2a2ff52 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:25.728617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11414b06-5f9a-4e29-ba71-65f9b2a2ff52 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:25.751951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:25.765294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:25.769226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.769226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.769226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.769226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:25.769226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:25.769653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:25.775792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 472/953] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:25.784585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:25.797723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:25.895771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421b959-c50d-4a4c-93ad-e8e447ce245d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:25.933268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256f3f6-34da-480f-bcf4-be82fc9d9143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:25.934141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f3f6-34da-480f-bcf4-be82fc9d9143 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.934584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f3f6-34da-480f-bcf4-be82fc9d9143 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.935028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f3f6-34da-480f-bcf4-be82fc9d9143 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.935868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256f3f6-34da-480f-bcf4-be82fc9d9143 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:25.936315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256f3f6-34da-480f-bcf4-be82fc9d9143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:25.937813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256f3f6-34da-480f-bcf4-be82fc9d9143 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:25.944399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 482/954] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:25.951961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:25.979990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d58371-e014-4203-947b-8e7fa3d955cc None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.980274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d58371-e014-4203-947b-8e7fa3d955cc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.980553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d58371-e014-4203-947b-8e7fa3d955cc None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:25.983110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 483/955] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:25.986915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184f1a3-3099-48f9-b777-94b70b4d0a45 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:25.987202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184f1a3-3099-48f9-b777-94b70b4d0a45 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:25.987480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184f1a3-3099-48f9-b777-94b70b4d0a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:26.006422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2184f1a3-3099-48f9-b777-94b70b4d0a45 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:26.007975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2184f1a3-3099-48f9-b777-94b70b4d0a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:26.074512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a44289-0713-4004-aa09-5227f4776f5e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:26.082981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 484/956] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:26.102507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184f1a3-3099-48f9-b777-94b70b4d0a45 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:26.106548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:26.116426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:26.117311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:26.117591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:26.117863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:26.118737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:26.120628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:26.148113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 473/957] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:26.222371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184f1a3-3099-48f9-b777-94b70b4d0a45 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:26.268956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:26.269615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:26.269893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:26.270164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:26.270803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:26.272780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:26.282915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:26.284084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:26.284952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:26.289986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 485/958] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:26.291438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:26.342146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:26.407844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2340dd-639e-41ee-bb41-424cf89d3abe None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:26.442093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514cd0-a507-47e2-aeab-f309596b6e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:26.442714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514cd0-a507-47e2-aeab-f309596b6e42 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:26.443025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514cd0-a507-47e2-aeab-f309596b6e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:26.443827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514cd0-a507-47e2-aeab-f309596b6e42 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:26.444498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88514cd0-a507-47e2-aeab-f309596b6e42 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:26.444802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88514cd0-a507-47e2-aeab-f309596b6e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:26.446167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88514cd0-a507-47e2-aeab-f309596b6e42 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:26.455004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 474/959] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:26.456967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:26.523819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:26.563211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e789043-f763-4450-9faf-2baf24c4b326 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:26.598268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34aef3-0bd9-48a7-a3d3-9b54c48acfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:26.598896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34aef3-0bd9-48a7-a3d3-9b54c48acfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:26.599206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34aef3-0bd9-48a7-a3d3-9b54c48acfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:26.599511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34aef3-0bd9-48a7-a3d3-9b54c48acfe4 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:26.600181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34aef3-0bd9-48a7-a3d3-9b54c48acfe4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:26.600485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34aef3-0bd9-48a7-a3d3-9b54c48acfe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:26.602001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34aef3-0bd9-48a7-a3d3-9b54c48acfe4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:26.610824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 486/960] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:27.252719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b339d-88b7-4c7f-b5c4-739d964d916f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.252719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b339d-88b7-4c7f-b5c4-739d964d916f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.252719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b339d-88b7-4c7f-b5c4-739d964d916f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.252719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 475/961] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:27.253593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e303bf-2717-407e-be30-1adb7552bc67 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.253593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e303bf-2717-407e-be30-1adb7552bc67 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.253593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e303bf-2717-407e-be30-1adb7552bc67 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.253593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 487/962] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:27.258897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb8f1d-e5b3-464a-b909-15339f8c6fc0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.258897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb8f1d-e5b3-464a-b909-15339f8c6fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.258897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb8f1d-e5b3-464a-b909-15339f8c6fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.262350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e2fab-edb1-4074-83a6-f1605cf8d61e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.262631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e2fab-edb1-4074-83a6-f1605cf8d61e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.262920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e2fab-edb1-4074-83a6-f1605cf8d61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.280398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1cbb8f1d-e5b3-464a-b909-15339f8c6fc0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:27.281594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1cbb8f1d-e5b3-464a-b909-15339f8c6fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:27.283095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8a2e2fab-edb1-4074-83a6-f1605cf8d61e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:27.284477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8a2e2fab-edb1-4074-83a6-f1605cf8d61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:27.355816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 476/963] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:27.372354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 488/964] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:27.380730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb8f1d-e5b3-464a-b909-15339f8c6fc0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:27.396119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e2fab-edb1-4074-83a6-f1605cf8d61e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:27.415980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:27.464029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:27.483701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb8f1d-e5b3-464a-b909-15339f8c6fc0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:27.513866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e2fab-edb1-4074-83a6-f1605cf8d61e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:27.519042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:27.519663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.519943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.520220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.520887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:27.522839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:27.525382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING keystone.server.flask.common [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 06 14:18:27.539478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.sql.core [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 14:18:27.539478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 14:18:27.539478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [parameters: {'user_id': '50329f9bdef54a768074af3d90f20b18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 14:18:27.539478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80150) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Aug 06 14:18:27.546787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 14:18:27.547927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application yield resource Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise self.value Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application session.commit() Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 14:18:27.549101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.flush() Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application compat.raise_( Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 14:18:27.550427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 14:18:27.551550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 14:18:27.552638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [parameters: {'user_id': '50329f9bdef54a768074af3d90f20b18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:27.553604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 06 14:18:27.554476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 06 14:18:27.555629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 06 14:18:27.555629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 06 14:18:27.555629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 14:18:27.555629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 14:18:27.555629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 14:18:27.555629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:27.555629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 477/965] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Aug 6 14:18:27 2022] POST /identity/v3/users => generated 160 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:27.557147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:27.563338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:27.564189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.564622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.565059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.576701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:27.582252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:27.583224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:27.584604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:27.591836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 489/966] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Aug 6 14:18:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:27.592871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:27.668383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:27.672490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72e72-ec7b-4686-b3a8-c67a5e3a27be None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:27.712572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:27.713319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.713699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.714080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.716381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4f267-185a-430f-86c3-79694861d613 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:27.722123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:27.724928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:27.737587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:27.738623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:27.739696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 14:18:27.740752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:27.745073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 478/967] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:27.750579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:27.765238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:27.765865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.766139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.766406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.767215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:27.767515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:27.768870 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:27.778533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 490/968] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Aug 6 14:18:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:27.787947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:27.889544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5698ec-0116-4a81-a265-4f9d7ec303c1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:27.905405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fbd3d-59b1-4f57-a81b-cb02d5f93896 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:27.938283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:27.938941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.939225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.939501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.940259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:27.940575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:27.942007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:27.946227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:27.947192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:27.947694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:27.948181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:27.961083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:27.962818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:27.980076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:27.980991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:27.981880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:27.981880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:27.981880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:27.981880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:27.984932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 491/970] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:27.986943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 479/970] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Aug 6 14:18:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:27.990941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.028917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.107455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998201cb-4021-4b31-bf9b-7a35b94ec3c4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.145223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3678693-c261-4758-bcfc-751091ccd7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.146266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3678693-c261-4758-bcfc-751091ccd7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.146763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3678693-c261-4758-bcfc-751091ccd7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.147304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3678693-c261-4758-bcfc-751091ccd7c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.148211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3678693-c261-4758-bcfc-751091ccd7c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:28.148727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3678693-c261-4758-bcfc-751091ccd7c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:28.150290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3678693-c261-4758-bcfc-751091ccd7c3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:28.163821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 492/971] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:28.169174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.191954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.267555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfefd7-8d70-4287-b050-fa90786438a2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.267904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfefd7-8d70-4287-b050-fa90786438a2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.268117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfefd7-8d70-4287-b050-fa90786438a2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.269738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 493/972] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:28.275329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35060898-4dd9-4764-b862-c05f3c5229ed None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.275614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35060898-4dd9-4764-b862-c05f3c5229ed None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.275875 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35060898-4dd9-4764-b862-c05f3c5229ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.282607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada875df-37a7-4302-b065-970eb22fa280 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.294993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-35060898-4dd9-4764-b862-c05f3c5229ed None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:28.296230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-35060898-4dd9-4764-b862-c05f3c5229ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:28.320146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.320891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.321076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.321348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.329181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:28.330616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:28.344889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:28.345867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:28.346815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 480/973] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:28.352088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.369439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 494/974] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:28.390912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35060898-4dd9-4764-b862-c05f3c5229ed None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.416797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.431012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.456000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a743d0b-ae9f-481d-a254-9b48da487e50 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.495455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.496071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.496353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.496633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.497405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:28.497818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:28.499229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:28.500523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35060898-4dd9-4764-b862-c05f3c5229ed None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.522365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 481/975] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:28.523413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.542728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.543345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.543627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.543902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.544520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:28.545872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:28.555261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:28.556311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:28.557144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 495/976] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:28.563253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.629299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba6317-1697-419e-8ac7-f3955a14d1f9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.653610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.665523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ce9d-c402-4424-9a62-01d0a8cc5e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.666260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ce9d-c402-4424-9a62-01d0a8cc5e16 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.666543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ce9d-c402-4424-9a62-01d0a8cc5e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.666821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ce9d-c402-4424-9a62-01d0a8cc5e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.669593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.669755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb36d5-c49c-4780-b7cb-7a7aad967730 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.704337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978c512-e497-4de7-ac5d-b318bfc0758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.704955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c512-e497-4de7-ac5d-b318bfc0758c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.705234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c512-e497-4de7-ac5d-b318bfc0758c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.705512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c512-e497-4de7-ac5d-b318bfc0758c None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.706177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c512-e497-4de7-ac5d-b318bfc0758c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:28.706477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c512-e497-4de7-ac5d-b318bfc0758c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:28.707856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c512-e497-4de7-ac5d-b318bfc0758c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:28.712974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 496/977] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:28.712974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c569431-c3d4-40a0-9af9-88025410e6ae None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.712974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c569431-c3d4-40a0-9af9-88025410e6ae None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.712974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c569431-c3d4-40a0-9af9-88025410e6ae None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.712974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 497/978] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:28.718441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c569431-c3d4-40a0-9af9-88025410e6ae None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.743499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ce9d-c402-4424-9a62-01d0a8cc5e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:28.744943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ce9d-c402-4424-9a62-01d0a8cc5e16 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:28.817170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c569431-c3d4-40a0-9af9-88025410e6ae None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.850708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 482/979] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:28.852112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258ace0-c2c0-426a-9927-eaf5e9ea0510 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.852376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258ace0-c2c0-426a-9927-eaf5e9ea0510 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.852651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258ace0-c2c0-426a-9927-eaf5e9ea0510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.856183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.856806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.857084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.857356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:28.858024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:28.858323 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:28.859695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:28.864689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 498/980] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:28.865313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.870016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0258ace0-c2c0-426a-9927-eaf5e9ea0510 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:28.871568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0258ace0-c2c0-426a-9927-eaf5e9ea0510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:28.883052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.898928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:28.938084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 483/981] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:28.947020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258ace0-c2c0-426a-9927-eaf5e9ea0510 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:28.966135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907d922-fb01-494d-a300-5025891f8404 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:28.998612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:28.999272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:28.999551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:28.999835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.010740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.012198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:29.026601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:29.027691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:29.028553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:29.028553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:29.028553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:29.028553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:29.028553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 499/982] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:29.028553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:29.049575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258ace0-c2c0-426a-9927-eaf5e9ea0510 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:29.084059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:29.084653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:29.084934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:29.085213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.085971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] RBAC: Adding query filter params (interface=admin, service_id=7ea5db7078494d5e9d2d8013d5cbc770, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:29.086282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.087683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.092880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 484/983] 10.222.0.17 () {56 vars in 1249 bytes} [Sat Aug 6 14:18:28 2022] GET /identity/v3/endpoints?service_id=7ea5db7078494d5e9d2d8013d5cbc770&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:29.093468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:29.100829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:29.121734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:29.136124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55828ed-c4b8-4ae1-b13b-ea8b9f01fc6f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:29.167952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:29.168560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:29.168838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:29.169111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.169784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.171158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.185623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 500/984] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:29 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:29.186397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:29.193694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b308854-8461-413d-b4a3-dcee56fcd8dd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:29.244131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:29.244773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:29.245049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:29.245321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.246018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:29.246325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.247747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.263040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 485/985] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:29.269022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:29.322524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977fb42-2af0-4112-b3f5-f4c90c650d3d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:29.350860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:29.364439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:29.366130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:29.366427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:29.366705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.367556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:29.367679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.369445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.377638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 501/986] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:29 2022] GET /identity/v3/services => generated 1727 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:18:29.386450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57ede-60cc-4dd3-ad5d-430132f1fdc2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:29.425352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:29.426012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:29.426290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:29.426561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.435216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.436706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:29.449292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:29.450262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:29.451209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 486/987] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:29.456818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:29.567949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b84f3-52fa-4cff-bd3a-7b19618f1e1b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:29.602435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1132a38-d7c1-4fe9-be5b-4cbeddc020a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:29.603006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132a38-d7c1-4fe9-be5b-4cbeddc020a5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:29.603286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132a38-d7c1-4fe9-be5b-4cbeddc020a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:29.603554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132a38-d7c1-4fe9-be5b-4cbeddc020a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.604289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132a38-d7c1-4fe9-be5b-4cbeddc020a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:29.604590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132a38-d7c1-4fe9-be5b-4cbeddc020a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.605995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132a38-d7c1-4fe9-be5b-4cbeddc020a5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.618915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 502/988] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:29.624588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:29.724230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccfcc3-d1fa-49ae-894f-9d6bddbbeadb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:29.757398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2610e-ea31-4a84-9315-5e3999a4ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:29.758011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2610e-ea31-4a84-9315-5e3999a4ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:29.758287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2610e-ea31-4a84-9315-5e3999a4ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:29.758556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2610e-ea31-4a84-9315-5e3999a4ebf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:29.759365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2610e-ea31-4a84-9315-5e3999a4ebf7 None admin] RBAC: Adding query filter params (role.id=7e2a69597b9f47a598002ebca7e97820, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=5f26e831b05740c9a1d510f107e74b57) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:29.759672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2610e-ea31-4a84-9315-5e3999a4ebf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:29.761092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2610e-ea31-4a84-9315-5e3999a4ebf7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:29.767082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 487/989] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:29 2022] GET /identity/v3/role_assignments?user.id=5f26e831b05740c9a1d510f107e74b57&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=7e2a69597b9f47a598002ebca7e97820 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:31.254287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ac37-e1c7-494f-aa63-2994cac377f1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.255131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ac37-e1c7-494f-aa63-2994cac377f1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.255131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ac37-e1c7-494f-aa63-2994cac377f1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.256520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 503/990] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:31.261853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829049b0-50ac-47cf-b443-1f55d33fd800 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.262279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829049b0-50ac-47cf-b443-1f55d33fd800 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.262448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829049b0-50ac-47cf-b443-1f55d33fd800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.280872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-829049b0-50ac-47cf-b443-1f55d33fd800 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:31.282054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-829049b0-50ac-47cf-b443-1f55d33fd800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:31.284218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:31.350931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 488/991] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:31.372817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534ac37-e1c7-494f-aa63-2994cac377f1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:31.404854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98ed9-ecf2-4304-a805-400b4775fccb None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.405144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98ed9-ecf2-4304-a805-400b4775fccb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.405426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98ed9-ecf2-4304-a805-400b4775fccb None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.406938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 489/992] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:31.411549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1826a65-219e-498e-a26c-6b7287ce6c5f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.412394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1826a65-219e-498e-a26c-6b7287ce6c5f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.412672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1826a65-219e-498e-a26c-6b7287ce6c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.426019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:31.432676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d1826a65-219e-498e-a26c-6b7287ce6c5f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:31.434456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d1826a65-219e-498e-a26c-6b7287ce6c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:31.480385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534ac37-e1c7-494f-aa63-2994cac377f1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:31.507975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 490/993] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:31.520565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:31.521709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.521985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.522254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.522900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:31.524235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:31.530306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1826a65-219e-498e-a26c-6b7287ce6c5f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:31.533100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:31.534082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:31.535340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 504/994] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:31.541563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:31.611177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:31.646326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1826a65-219e-498e-a26c-6b7287ce6c5f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:31.652080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a693d-1a75-4266-a326-461f28ec8f28 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:31.667132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:31.684132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:31.684780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.685058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.685328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.685951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:31.687352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:31.690780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:31.691424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.691700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.691970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.692633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:31.692954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:31.694277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:31.696277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:31.697227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:31.698045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 491/995] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Aug 6 14:18:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:31.705771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 505/996] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:31.705909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:31.706582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:31.813863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16939997-c62a-4ed0-8f91-9d80e747cecb None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:31.817309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583a71a-aa56-4397-87b5-70e06e56cf89 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:31.853704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:31.853963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:31.854424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.854712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.855131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.855798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:31.856078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:31.857563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:31.860244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:31.860910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:31.861161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:31.861434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:31.862104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:31.862404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:31.867850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 506/997] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:31.867978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:31.869266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 492/998] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:31.870424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:31.876248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:31.918055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:31.974884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947d67b-7ed6-4412-a7d0-17e42ed61453 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:31.976995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6b4ac-3632-49e7-81a5-a272d866cabf None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:32.007710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bec273-8483-42a1-b3e8-877f415a3850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:32.008321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bec273-8483-42a1-b3e8-877f415a3850 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.008604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bec273-8483-42a1-b3e8-877f415a3850 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.008881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bec273-8483-42a1-b3e8-877f415a3850 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.009106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a88de-2300-49ef-9ea9-d01466e30809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:32.009629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bec273-8483-42a1-b3e8-877f415a3850 None admin] RBAC: Adding query filter params (interface=public, service_id=564e239c1ae04e939efe5fd122a73db2, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:32.009779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a88de-2300-49ef-9ea9-d01466e30809 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.009941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bec273-8483-42a1-b3e8-877f415a3850 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:32.010064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a88de-2300-49ef-9ea9-d01466e30809 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.010340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a88de-2300-49ef-9ea9-d01466e30809 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.011042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a88de-2300-49ef-9ea9-d01466e30809 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:32.011336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bec273-8483-42a1-b3e8-877f415a3850 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:32.011437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a88de-2300-49ef-9ea9-d01466e30809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:32.012691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a88de-2300-49ef-9ea9-d01466e30809 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:32.016361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 507/999] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:18:31 2022] GET /identity/v3/endpoints?service_id=564e239c1ae04e939efe5fd122a73db2&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:32.017562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 493/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Aug 6 14:18:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:32.017673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1dad-1bea-45c2-aaf8-0c11d18b449f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.017757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1dad-1bea-45c2-aaf8-0c11d18b449f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.018025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1dad-1bea-45c2-aaf8-0c11d18b449f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.019365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 508/1001] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:32.025037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd6a6-cf34-48f3-911b-8f19b1a8d9d0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.025580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd6a6-cf34-48f3-911b-8f19b1a8d9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.025580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd6a6-cf34-48f3-911b-8f19b1a8d9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.026608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b1dad-1bea-45c2-aaf8-0c11d18b449f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:32.043813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b1fcd6a6-cf34-48f3-911b-8f19b1a8d9d0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:32.045005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b1fcd6a6-cf34-48f3-911b-8f19b1a8d9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:32.098822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:32.110584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 494/1002] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:32.111430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcd6a6-cf34-48f3-911b-8f19b1a8d9d0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:32.129266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b1dad-1bea-45c2-aaf8-0c11d18b449f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:32.136705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:32.165636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:32.166246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.166524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.166792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.167526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:32.168871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:32.184837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 509/1003] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:18:32.185553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:32.224030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcd6a6-cf34-48f3-911b-8f19b1a8d9d0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:32.272558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:32.273253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.273456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.273739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.274396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:32.275817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:32.283210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 495/1004] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:32 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:18:32.320073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f556b08-5daa-466f-be7d-3c12c02d6302 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:32.360808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:32.361388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.361674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.361953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] PATH_INFO: `/v3/services/image` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.362595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:32.364029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:32.374627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:32.375728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:32.376739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 510/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Aug 6 14:18:32 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:32.382274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:32.439931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:32.495599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b3794-7ddf-4483-8a6f-64beb6427e35 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:32.529290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4b735-7dee-40ae-aff0-0494f3e5f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:32.529915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4b735-7dee-40ae-aff0-0494f3e5f8bb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.530198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4b735-7dee-40ae-aff0-0494f3e5f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.530468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4b735-7dee-40ae-aff0-0494f3e5f8bb None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.531161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4b735-7dee-40ae-aff0-0494f3e5f8bb None admin] RBAC: Adding query filter params (name=image) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:32.531467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4b735-7dee-40ae-aff0-0494f3e5f8bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:32.532814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4b735-7dee-40ae-aff0-0494f3e5f8bb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:32.537825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 496/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:32.543440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:32.578970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:32.657493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5269aba-a6d3-4dbb-a26a-8dd6bf031a98 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:32.693529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1befdf0-e329-4d2e-9f07-0d1ef40fb25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:32.694561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befdf0-e329-4d2e-9f07-0d1ef40fb25c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:32.695090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befdf0-e329-4d2e-9f07-0d1ef40fb25c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:32.695582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befdf0-e329-4d2e-9f07-0d1ef40fb25c None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:32.696793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befdf0-e329-4d2e-9f07-0d1ef40fb25c None admin] RBAC: Adding query filter params (type=image) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:32.697139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befdf0-e329-4d2e-9f07-0d1ef40fb25c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:32.698811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befdf0-e329-4d2e-9f07-0d1ef40fb25c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:32.710966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 511/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:34.256216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440059c1-6212-4871-8a72-64efd9859d07 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.256767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440059c1-6212-4871-8a72-64efd9859d07 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.256767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440059c1-6212-4871-8a72-64efd9859d07 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.259082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 497/1008] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:34.264222 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7095a2-b6b1-4160-bcdb-995b7682d663 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.264634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7095a2-b6b1-4160-bcdb-995b7682d663 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.264797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7095a2-b6b1-4160-bcdb-995b7682d663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.282998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5b7095a2-b6b1-4160-bcdb-995b7682d663 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:34.283809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbfbe6-7432-40f6-afeb-3db148470592 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.284096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbfbe6-7432-40f6-afeb-3db148470592 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.284209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5b7095a2-b6b1-4160-bcdb-995b7682d663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:34.284381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbfbe6-7432-40f6-afeb-3db148470592 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.285827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 498/1009] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:34.290470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec44c6a-0d18-4112-ab3e-d9ed7b9387ba None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.290748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec44c6a-0d18-4112-ab3e-d9ed7b9387ba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.291047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec44c6a-0d18-4112-ab3e-d9ed7b9387ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.309548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1ec44c6a-0d18-4112-ab3e-d9ed7b9387ba None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:34.310714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1ec44c6a-0d18-4112-ab3e-d9ed7b9387ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:34.355534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 512/1010] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:34.378488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7095a2-b6b1-4160-bcdb-995b7682d663 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:34.384578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 499/1011] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:34.403689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:34.406355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec44c6a-0d18-4112-ab3e-d9ed7b9387ba None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:34.454099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:34.483398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7095a2-b6b1-4160-bcdb-995b7682d663 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:34.517500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec44c6a-0d18-4112-ab3e-d9ed7b9387ba None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:34.517677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:34.518106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.518384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.518658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.519835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:34.521267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:34.522855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 513/1012] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:34.528444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:34.554614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:34.555653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.556183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.556635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.557433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:34.559223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:34.569172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:34.570186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 14:18:34.571260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:34.578152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 500/1013] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:34.579272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:34.621333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:34.631668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94195be0-d3ad-4980-99b5-cadde9b05c5a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:34.668989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8e9e5-dcff-4b7d-a023-eea452e7fadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:34.669641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e9e5-dcff-4b7d-a023-eea452e7fadc None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.669969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e9e5-dcff-4b7d-a023-eea452e7fadc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.670520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e9e5-dcff-4b7d-a023-eea452e7fadc None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.671675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8e9e5-dcff-4b7d-a023-eea452e7fadc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:34.673114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8e9e5-dcff-4b7d-a023-eea452e7fadc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:34.682076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936d3d4-ddf7-43af-a4d7-573be9c16ce3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:34.691237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:34.709862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 514/1014] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:34 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:18:34.709862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe81c3-a128-4118-a45c-03207f033559 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.709862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe81c3-a128-4118-a45c-03207f033559 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.709862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe81c3-a128-4118-a45c-03207f033559 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.709862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 515/1015] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:34.712844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71663452-2ac2-4919-849c-558eff6eb693 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.713327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71663452-2ac2-4919-849c-558eff6eb693 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.713837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71663452-2ac2-4919-849c-558eff6eb693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.732279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:34.732923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.733204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.733480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.734166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:34.734473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:34.735850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-71663452-2ac2-4919-849c-558eff6eb693 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:34.735969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:34.737084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-71663452-2ac2-4919-849c-558eff6eb693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:34.748366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 501/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:34.750330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:34.814523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 516/1017] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:34.837077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71663452-2ac2-4919-849c-558eff6eb693 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:34.873540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcea833-a145-40b5-90d1-d7c892964d73 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:34.913953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:34.914574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.916765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.916765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.916765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:34.916765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:34.917452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:34.922375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 502/1018] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Aug 6 14:18:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:34.928769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:34.929980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:34.955593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71663452-2ac2-4919-849c-558eff6eb693 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:34.964323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:34.990554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953774de-1361-4c4f-bec1-2447298ee0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:34.991204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953774de-1361-4c4f-bec1-2447298ee0e5 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:34.991490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953774de-1361-4c4f-bec1-2447298ee0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:34.991764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953774de-1361-4c4f-bec1-2447298ee0e5 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:34.992454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953774de-1361-4c4f-bec1-2447298ee0e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:34.993808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953774de-1361-4c4f-bec1-2447298ee0e5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:35.011477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 517/1019] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:18:34 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:18:35.035963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bebbe-edf0-47ef-b172-682b377ee1e7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:35.081000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65372a1-10b8-4e89-b580-d2f522f644d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:35.081610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65372a1-10b8-4e89-b580-d2f522f644d1 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:35.081886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65372a1-10b8-4e89-b580-d2f522f644d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:35.082157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65372a1-10b8-4e89-b580-d2f522f644d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:35.082967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65372a1-10b8-4e89-b580-d2f522f644d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:35.084926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65372a1-10b8-4e89-b580-d2f522f644d1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:35.114906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 503/1020] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:18:36.754793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0efd-7c99-4a7b-b069-d4af69d71b26 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:36.755361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0efd-7c99-4a7b-b069-d4af69d71b26 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:36.755478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0efd-7c99-4a7b-b069-d4af69d71b26 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:36.756903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 518/1021] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:36.761976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e676-0f1d-4235-8993-d1aa546d4e3b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:36.762363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e676-0f1d-4235-8993-d1aa546d4e3b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:36.762521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e676-0f1d-4235-8993-d1aa546d4e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:36.780800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d402e676-0f1d-4235-8993-d1aa546d4e3b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:36.781988 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d402e676-0f1d-4235-8993-d1aa546d4e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:36.864746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 504/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:36.889356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb0efd-7c99-4a7b-b069-d4af69d71b26 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:36.996968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb0efd-7c99-4a7b-b069-d4af69d71b26 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.020785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb327-81fe-4094-ad33-ebe7fa448e61 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.021597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb327-81fe-4094-ad33-ebe7fa448e61 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.021877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb327-81fe-4094-ad33-ebe7fa448e61 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.023448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 505/1023] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:37.028322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80eec14-490d-4411-8c24-b7fb70eb07d6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.028608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80eec14-490d-4411-8c24-b7fb70eb07d6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.028888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80eec14-490d-4411-8c24-b7fb70eb07d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.037552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.038197 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.038476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.038747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.039931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.041344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.043091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 519/1024] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:37.046665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d80eec14-490d-4411-8c24-b7fb70eb07d6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:37.047879 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d80eec14-490d-4411-8c24-b7fb70eb07d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:37.048499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.051557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.058205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.131167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 506/1025] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:37.142048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80eec14-490d-4411-8c24-b7fb70eb07d6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.158617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2cbf9-2c6a-4923-84b7-d18fdfb87c6a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.196180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b2e40-e9d4-41d5-a276-edde3d429ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.196898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2e40-e9d4-41d5-a276-edde3d429ff1 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.197187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2e40-e9d4-41d5-a276-edde3d429ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.197465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2e40-e9d4-41d5-a276-edde3d429ff1 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.198146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b2e40-e9d4-41d5-a276-edde3d429ff1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.199610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b2e40-e9d4-41d5-a276-edde3d429ff1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.225103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 520/1026] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:18:37 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:18:37.226360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ea027-fc72-4362-b8ca-77fb4d4ce0d9 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.226650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ea027-fc72-4362-b8ca-77fb4d4ce0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.226952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ea027-fc72-4362-b8ca-77fb4d4ce0d9 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.228181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 521/1027] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:37.235023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5707-f6df-4335-86e6-c760a11f239a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.235361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5707-f6df-4335-86e6-c760a11f239a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.235673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5707-f6df-4335-86e6-c760a11f239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.249582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80eec14-490d-4411-8c24-b7fb70eb07d6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.256236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9b0e5707-f6df-4335-86e6-c760a11f239a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:37.257468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9b0e5707-f6df-4335-86e6-c760a11f239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:37.288874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.289444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.289779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.289997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.290638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.292068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:37.301838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:37.303536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:37.304398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 507/1028] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:37.309754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.321873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.322509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.339907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 522/1029] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:37.365251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5707-f6df-4335-86e6-c760a11f239a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.421943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e53e4-24a1-4be1-ba7f-3c696d5310f3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.461247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.461853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.462133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.462408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.463105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] RBAC: Adding query filter params (name=image) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:37.463405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.464763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.473304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 508/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:37.479214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.483214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5707-f6df-4335-86e6-c760a11f239a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.521089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.521734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.522016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.522293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.532870 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:37.537770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:37.538951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:37.539816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 523/1031] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:37.544874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.559224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.592477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168bd47-063d-4bfc-b93c-ff6e1ac1fb5d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.597637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.631619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.632311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.632663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.632944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.633617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] RBAC: Adding query filter params (type=image) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:37.633910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.635307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.640366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 509/1032] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:37.646283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.654950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9c1cb-5f86-4075-bd6d-2b2ea87a34a2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.690763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.691415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.691691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.691966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.692675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:37.692976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.694330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.703147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 524/1033] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:37.708762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.765117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd27076-1af0-409f-883b-7a1235844556 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.801748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.803006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.803433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.803708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.804448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] RBAC: Adding query filter params (interface=public, service_id=ffd35aa199b245e4a526902d0edaccfe, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:37.804763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.806240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.808776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.813045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 510/1034] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/endpoints?service_id=ffd35aa199b245e4a526902d0edaccfe&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:37.818097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.818502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd4474-834b-4a5f-9d92-0cc5c1ec0ce0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.829953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:37.861951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.862824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.863322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.864192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.877830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.879876 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:37.896178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:37.897163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:37.898039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:37.898039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:37.898039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:37.898039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:37.898039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 525/1035] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:37.903020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:37.928006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbe2aa-5e50-4ece-ab7b-c824a9f6995f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:37.963611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:37.964335 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:37.964653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:37.964932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:37.965580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:37.967033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:37.979265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 511/1036] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:18:38.006085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93f2a6-9dc2-4007-952a-64e8403a323b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:38.043932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:38.044574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:38.044855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:38.045131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:38.045869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:38.046178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:38.047588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:38.065816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 526/1037] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:38.067537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:38.133802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:38.173990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c226a-681d-4e13-91fb-8510499d5e4f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:38.208110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:38.209264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:38.209541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:38.209808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:38.217127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:38.218548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:38.230076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:38.231319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:38.232270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 512/1038] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:18:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:38.238698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:38.263909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:38.341706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb519ac-3a91-4aef-95a8-7936fa697bda None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:38.375682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:38.376301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:38.376581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:38.376858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:38.377645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:38.377957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:38.379481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:38.403262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 527/1039] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:18:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:38.404783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:38.507030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d70b3d-6c50-4184-b812-cbd25af477b5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:38.529676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:38.541470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:38.542144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:38.542420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:38.542688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:38.608813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:38.610201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:38.709752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 513/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:38.713087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:38.795090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:38.815812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f6164-79e7-420e-8afe-6b37c0addaaf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:38.857670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:38.858318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:38.858594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:38.858926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:38.872648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:38.874076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:38.889243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:38.890127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:38.896054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:38.896054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:38.896054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:38.896054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:38.896054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 528/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:38.896538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:38.922115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:38.997244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b41f-44e8-4f38-b0b5-d6694d1c042e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:39.030605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:39.031258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.031538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.031812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.032492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:39.032801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:39.034164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:39.049745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 514/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:39.051598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:39.154255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9786ae-59a0-4331-8eff-5ae0fcbdb950 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:39.184165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:39.190426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21374f0-78c3-4a0e-93c8-ff9f67dc2d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:39.191092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21374f0-78c3-4a0e-93c8-ff9f67dc2d56 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.191375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21374f0-78c3-4a0e-93c8-ff9f67dc2d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.191647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21374f0-78c3-4a0e-93c8-ff9f67dc2d56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.198705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21374f0-78c3-4a0e-93c8-ff9f67dc2d56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:39.200173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21374f0-78c3-4a0e-93c8-ff9f67dc2d56 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d21374f0-78c3-4a0e-93c8-ff9f67dc2d56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:39.211411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:39.212269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:39.217842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 529/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:39.218325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:39.305268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:39.305455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fbc21-ca7a-4d49-9ff4-ac3b5642633e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.305455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fbc21-ca7a-4d49-9ff4-ac3b5642633e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.305455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fbc21-ca7a-4d49-9ff4-ac3b5642633e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.305455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 530/1044] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:39.310236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4282643-aff8-45c3-b93c-500324b0d1f2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.310587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4282643-aff8-45c3-b93c-500324b0d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.311087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4282643-aff8-45c3-b93c-500324b0d1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.323146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbb4ef-058e-4b47-ac04-506cd560effd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:39.330900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e4282643-aff8-45c3-b93c-500324b0d1f2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:39.332744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e4282643-aff8-45c3-b93c-500324b0d1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:39.357381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:39.357993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.358278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.358549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.359320 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:39.359626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:39.361054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:39.373521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 515/1045] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:39.379328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:39.410969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 531/1046] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:39.435002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4282643-aff8-45c3-b93c-500324b0d1f2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:39.480862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ee999-2792-4e47-adb0-469e4248d38f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:39.515964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:39.516864 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.517143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.517415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.518370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] RBAC: Adding query filter params (role.id=7e2a69597b9f47a598002ebca7e97820, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=075e9c0ef8fd406087d05a6ce36dc4e2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:39.519080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:39.520514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:39.526409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 516/1047] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:39 2022] GET /identity/v3/role_assignments?user.id=075e9c0ef8fd406087d05a6ce36dc4e2&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=7e2a69597b9f47a598002ebca7e97820 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:39.549070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4282643-aff8-45c3-b93c-500324b0d1f2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:39.574897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:39.589344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:39.590001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.590279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.590549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.602807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:39.607787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:39.608767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:39.609627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 532/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:39.618921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:39.630611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:39.726714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd5aa-f539-41e7-9459-36825ba5ce0a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:39.759543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:39.760228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.760521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.760799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.761509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:39.761817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:39.763226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:39.777938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 517/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:39.779469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:39.888184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090b7d-c4c0-4b22-9e7e-5213aa5638e1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:39.929596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc31c7a-a1a8-45d6-8f47-e11b0ec292a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:39.930235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc31c7a-a1a8-45d6-8f47-e11b0ec292a2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:39.930511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc31c7a-a1a8-45d6-8f47-e11b0ec292a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:39.930781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc31c7a-a1a8-45d6-8f47-e11b0ec292a2 None admin] PATH_INFO: `/v3/users/f2a74b17c62545659dff3dcc0befe61e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:39.945234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc31c7a-a1a8-45d6-8f47-e11b0ec292a2 None admin] RBAC: Authorizing `identity:get_user(user_id=f2a74b17c62545659dff3dcc0befe61e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:39.946654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc31c7a-a1a8-45d6-8f47-e11b0ec292a2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:39.952993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 533/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:39 2022] GET /identity/v3/users/f2a74b17c62545659dff3dcc0befe61e => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:39.954187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:40.008579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b676-2941-432a-86bb-70127aa147e3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.008819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b676-2941-432a-86bb-70127aa147e3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.009090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b676-2941-432a-86bb-70127aa147e3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.010741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 534/1051] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:40.015326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136e112-df4d-4fb3-b595-a1513ae2a0f8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.015614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136e112-df4d-4fb3-b595-a1513ae2a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.015890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136e112-df4d-4fb3-b595-a1513ae2a0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.028667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:40.035040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1136e112-df4d-4fb3-b595-a1513ae2a0f8 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:40.036232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1136e112-df4d-4fb3-b595-a1513ae2a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:40.061773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024cca8-7f9d-4829-a92d-8cb4e12be127 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:40.071790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:40.099362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:40.099980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.100265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.100536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.106696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 535/1052] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:40.129252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136e112-df4d-4fb3-b595-a1513ae2a0f8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:40.170155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:40.171631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:40.237775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136e112-df4d-4fb3-b595-a1513ae2a0f8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:40.238995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:40.275465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:40.276005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.282214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.282328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 518/1053] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 14:18:40.282410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.282410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:40.282410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:40.283661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:40.294254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:40.296686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:40.297572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 536/1054] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Aug 6 14:18:40 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:40.303701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:40.340533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:40.406037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346ac6f-8198-4a38-a970-caf1409326e3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:40.415163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a208b6-4b04-49d6-beca-03be9d73bea6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:40.442127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:40.443316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.443627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.443899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.445063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] RBAC: Authorizing `identity:get_project(project_id=ff54925983c24deeaf71b79a691b9bc1)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:40.446503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:40.448187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 519/1055] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:18:40 2022] GET /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:40.454372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:40.454845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:40.455492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.455771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.456046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.456851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] RBAC: Adding query filter params (name=image) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:40.457035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:40.458405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:40.463547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 537/1056] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:40.469699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:40.501308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:40.564950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e7fd1-44d4-4dfb-9a94-1d9d0ab88366 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:40.574774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:40.579525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054d606-71f8-4d9b-a258-9776a4e89f4e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:40.600069 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:40.600706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.600989 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.601257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.602058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=ff54925983c24deeaf71b79a691b9bc1, user.id=f2a74b17c62545659dff3dcc0befe61e) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:40.602364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:40.603873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:40.618907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 520/1057] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:40 2022] GET /identity/v3/role_assignments?user.id=f2a74b17c62545659dff3dcc0befe61e&scope.project.id=ff54925983c24deeaf71b79a691b9bc1&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:40.626109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e9d0b-4beb-49a8-a1bd-b73f984494fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:40.626759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e9d0b-4beb-49a8-a1bd-b73f984494fb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.627088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e9d0b-4beb-49a8-a1bd-b73f984494fb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.627369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e9d0b-4beb-49a8-a1bd-b73f984494fb None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.628069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e9d0b-4beb-49a8-a1bd-b73f984494fb None admin] RBAC: Adding query filter params (type=image) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:40.628380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e9d0b-4beb-49a8-a1bd-b73f984494fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:40.629829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e9d0b-4beb-49a8-a1bd-b73f984494fb None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:40.636474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 538/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:40.642572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:40.768615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26eea6-95b3-4d13-b772-9125c72af4d0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:40.770348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:40.805383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7d42c-4f4e-4e69-92a2-ab00e70f9169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:40.806002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7d42c-4f4e-4e69-92a2-ab00e70f9169 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:40.806280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7d42c-4f4e-4e69-92a2-ab00e70f9169 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:40.806550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7d42c-4f4e-4e69-92a2-ab00e70f9169 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:40.807296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7d42c-4f4e-4e69-92a2-ab00e70f9169 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:40.808659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7d42c-4f4e-4e69-92a2-ab00e70f9169 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:40.834982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 521/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:18:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:41.522887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08466c0-4f85-496c-9a1d-efed2a0c9edd None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:41.523415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08466c0-4f85-496c-9a1d-efed2a0c9edd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:41.523415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08466c0-4f85-496c-9a1d-efed2a0c9edd None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:41.524818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 539/1060] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:41.529847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972951c-652b-44d8-bae9-bfc5c940b70e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:41.530269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972951c-652b-44d8-bae9-bfc5c940b70e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:41.530446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972951c-652b-44d8-bae9-bfc5c940b70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:41.548798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f972951c-652b-44d8-bae9-bfc5c940b70e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:41.550226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f972951c-652b-44d8-bae9-bfc5c940b70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:41.623946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 522/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:41.645649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08466c0-4f85-496c-9a1d-efed2a0c9edd None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:41.750535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08466c0-4f85-496c-9a1d-efed2a0c9edd None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:41.784460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:41.785032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:41.785313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:41.785585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:41.796238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:41.801372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:41.802809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:41.803674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 540/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Aug 6 14:18:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:41.808957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972951c-652b-44d8-bae9-bfc5c940b70e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:41.904514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:41.922241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972951c-652b-44d8-bae9-bfc5c940b70e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:41.956565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:41.957184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:41.957465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:41.957740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:41.958473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:41.958779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:41.960176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:41.972671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 523/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Aug 6 14:18:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:41.973717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.005857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:42.085978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1e990-9ea2-4178-8d34-30bec53e4145 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:42.118940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:42.119576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.119858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.120133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.131077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:42.132513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:42.146484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:42.147542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:42.153095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:42.153095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:42.153095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:42.153095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:42.153095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 541/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:42.153389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.253212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304e1c1-dec9-4dd0-a062-4660a5784e99 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:42.286210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:42.287371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.287652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.287925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.288606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:42.288909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:42.290268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:42.302941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 524/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:42.308621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.384379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:42.410748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be819-f670-4192-a8be-f2abda8af6ea None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:42.445377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f3259-68e3-4066-b32f-d9ca05bcc568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:42.446019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f3259-68e3-4066-b32f-d9ca05bcc568 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.446300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f3259-68e3-4066-b32f-d9ca05bcc568 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.446577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f3259-68e3-4066-b32f-d9ca05bcc568 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.454113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f3259-68e3-4066-b32f-d9ca05bcc568 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:42.455569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f3259-68e3-4066-b32f-d9ca05bcc568 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-8c4f3259-68e3-4066-b32f-d9ca05bcc568 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:42.466804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:42.467864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:42.468807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:42.471229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 542/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:18:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:42.474452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.483895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:42.581190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacce2e2-ae52-46e7-b9dd-840438a797dc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:42.616643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:42.617266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.617550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.617829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.618614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:42.618959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:42.620390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:42.631549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb510f-469a-4141-a621-3474e8be2bfe None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.632030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb510f-469a-4141-a621-3474e8be2bfe None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.632132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb510f-469a-4141-a621-3474e8be2bfe None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.633616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 543/1067] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:42.638694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46325983-bce1-4c79-a632-2e8f06857c05 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.638818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 525/1068] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:18:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:42.639001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46325983-bce1-4c79-a632-2e8f06857c05 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.639097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46325983-bce1-4c79-a632-2e8f06857c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.644656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.658185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-46325983-bce1-4c79-a632-2e8f06857c05 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:42.659417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-46325983-bce1-4c79-a632-2e8f06857c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:42.709529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:42.733844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 544/1069] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:42.750446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25605dc-cfef-4563-87c9-8547bf207cd2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:42.755790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46325983-bce1-4c79-a632-2e8f06857c05 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.786343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:42.787526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.787804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.788068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.800479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:42.802282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:42.818070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:42.819007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:42.819941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:42.819941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:42.819941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:42.819941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:42.819941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 526/1070] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:42.823170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:42.825327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.867864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46325983-bce1-4c79-a632-2e8f06857c05 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:42.901916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3513c9-a6eb-4ff5-8be2-2e3e301685e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:42.902478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3513c9-a6eb-4ff5-8be2-2e3e301685e0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.902760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3513c9-a6eb-4ff5-8be2-2e3e301685e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.903067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3513c9-a6eb-4ff5-8be2-2e3e301685e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.913648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3513c9-a6eb-4ff5-8be2-2e3e301685e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-df3513c9-a6eb-4ff5-8be2-2e3e301685e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:42.918503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:42.919580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:42.920419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 545/1071] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:42.921105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a286-1206-4e8c-b5ce-3b61f7b7c461 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.921173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a286-1206-4e8c-b5ce-3b61f7b7c461 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.921607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a286-1206-4e8c-b5ce-3b61f7b7c461 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.922827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 546/1072] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:42.925639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832a286-1206-4e8c-b5ce-3b61f7b7c461 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:42.937518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3946830-dbe0-4ba4-9539-0d2594f30092 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:42.953590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:42.972012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29353aa9-b26b-4d0c-9f69-cb8e2b25b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:42.972627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353aa9-b26b-4d0c-9f69-cb8e2b25b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.972903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353aa9-b26b-4d0c-9f69-cb8e2b25b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.973173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353aa9-b26b-4d0c-9f69-cb8e2b25b2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:42.973874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29353aa9-b26b-4d0c-9f69-cb8e2b25b2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:42.974175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29353aa9-b26b-4d0c-9f69-cb8e2b25b2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:42.975563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29353aa9-b26b-4d0c-9f69-cb8e2b25b2a8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:42.987008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 527/1073] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:42.992971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a530c7-2e4f-4cc3-8f8c-092f7730a721 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:42.993258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a530c7-2e4f-4cc3-8f8c-092f7730a721 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:42.993536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a530c7-2e4f-4cc3-8f8c-092f7730a721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.010798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b2a530c7-2e4f-4cc3-8f8c-092f7730a721 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:43.012040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b2a530c7-2e4f-4cc3-8f8c-092f7730a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:43.027790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832a286-1206-4e8c-b5ce-3b61f7b7c461 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.050325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.061829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.062457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.062716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.063019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.063725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:43.064023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.065365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.075273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 547/1074] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:43.076061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.082482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 528/1075] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:43.083945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a530c7-2e4f-4cc3-8f8c-092f7730a721 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.180805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b735fc-5875-4a69-aed9-b13385852725 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.184026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a530c7-2e4f-4cc3-8f8c-092f7730a721 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.191267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.214810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.215494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.215775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.216048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.219314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.219948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.220224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.220496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] PATH_INFO: `/v3/users/f2a74b17c62545659dff3dcc0befe61e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.221662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] RBAC: Authorizing `identity:get_user(user_id=f2a74b17c62545659dff3dcc0befe61e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.222627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.223101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.224113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.224691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 529/1076] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:43 2022] GET /identity/v3/users/f2a74b17c62545659dff3dcc0befe61e => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:43.225436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:43.235418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:43.236501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 548/1077] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:43.242585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.289102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.324605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962331cf-f327-4a1b-a13d-baf83687407e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.342226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804831ed-3a93-4f17-bccd-234248c47fa5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.356305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.356921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.357197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.357465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.375059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.375747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.376026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.376295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.377454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] RBAC: Authorizing `identity:get_project(project_id=ff54925983c24deeaf71b79a691b9bc1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.378858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.381088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 549/1078] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:18:43 2022] GET /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:43.381832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.403548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.428892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.430335 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.479644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195b442-c082-47a0-9f91-42b5c5f95f2f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.493531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.514117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.514727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.515040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.515313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.516037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:43.516338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.517735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.531358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 530/1079] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:43.531358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.531598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 550/1080] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:43.532326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.600545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.624873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f920f0-d900-4159-bd9a-faceb5f07fb3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.629766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e047c04-4468-49ed-ac7a-658d44d73993 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.658152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.659057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.659057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.659347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1/users/f2a74b17c62545659dff3dcc0befe61e/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.661825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff54925983c24deeaf71b79a691b9bc1, user_id=f2a74b17c62545659dff3dcc0befe61e, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.663299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.664057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.664693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.664973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.665243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.676070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.676657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 531/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:18:43 2022] PUT /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1/users/f2a74b17c62545659dff3dcc0befe61e/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:18:43.677838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.678199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:43.711936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:43.712973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:43.714062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:43.714062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:43.714062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:43.714062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:18:43.714062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 551/1082] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:43.719261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:43.753120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.797784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094588-727b-49b2-a62b-21d3a49cb901 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.839259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89b6ba-61e0-43f0-a64f-c2b1a4152018 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:43.842679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df70be-5dc5-49ef-bab0-01432dd7ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.843799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df70be-5dc5-49ef-bab0-01432dd7ebf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.843799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df70be-5dc5-49ef-bab0-01432dd7ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.844000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df70be-5dc5-49ef-bab0-01432dd7ebf9 None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/075e9c0ef8fd406087d05a6ce36dc4e2/roles/7e2a69597b9f47a598002ebca7e97820` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.846392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df70be-5dc5-49ef-bab0-01432dd7ebf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=075e9c0ef8fd406087d05a6ce36dc4e2, role_id=7e2a69597b9f47a598002ebca7e97820)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.847877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df70be-5dc5-49ef-bab0-01432dd7ebf9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.850782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:43.864552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 532/1083] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:18:43 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/075e9c0ef8fd406087d05a6ce36dc4e2/roles/7e2a69597b9f47a598002ebca7e97820 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:18:43.881370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79702f1c-2fa8-4e3c-824c-9152e7d1524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:43.882064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79702f1c-2fa8-4e3c-824c-9152e7d1524e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:43.882348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79702f1c-2fa8-4e3c-824c-9152e7d1524e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:43.882618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79702f1c-2fa8-4e3c-824c-9152e7d1524e None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:43.883425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79702f1c-2fa8-4e3c-824c-9152e7d1524e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:43.883732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79702f1c-2fa8-4e3c-824c-9152e7d1524e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:43.885190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79702f1c-2fa8-4e3c-824c-9152e7d1524e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:43.906914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 552/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:45.691880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1a715-ff7b-42dc-8871-ac8252859e9c None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:45.692439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1a715-ff7b-42dc-8871-ac8252859e9c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:45.692439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1a715-ff7b-42dc-8871-ac8252859e9c None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:45.694355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 533/1085] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:45.699475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643930-0a64-4232-951c-6d40211a78f0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:45.699893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643930-0a64-4232-951c-6d40211a78f0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:45.700036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643930-0a64-4232-951c-6d40211a78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:45.717555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-62643930-0a64-4232-951c-6d40211a78f0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:45.718763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-62643930-0a64-4232-951c-6d40211a78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:45.784195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 553/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:45.807824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1a715-ff7b-42dc-8871-ac8252859e9c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:45.859903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb46b-2140-4d6a-84ac-092421996fc1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:45.860477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb46b-2140-4d6a-84ac-092421996fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:45.860764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb46b-2140-4d6a-84ac-092421996fc1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:45.862108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 554/1087] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:18:45.867815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a481d-dd4a-4998-93a2-61eacfe993c8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:45.868100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a481d-dd4a-4998-93a2-61eacfe993c8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:45.868377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a481d-dd4a-4998-93a2-61eacfe993c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:45.886835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-167a481d-dd4a-4998-93a2-61eacfe993c8 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:45.888048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-167a481d-dd4a-4998-93a2-61eacfe993c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:45.898462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:45.910934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1a715-ff7b-42dc-8871-ac8252859e9c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:45.944824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:45.945393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:45.945670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:45.945942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:45.954102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 555/1088] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:18:45.955902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:45.960847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:45.961931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:18:45.962991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:45.967148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 534/1089] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:18:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:45.967993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a481d-dd4a-4998-93a2-61eacfe993c8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:45.977431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.031220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.064943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a481d-dd4a-4998-93a2-61eacfe993c8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.076574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7eb84-2605-49fd-a225-9780e56c47a3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.097407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.098022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.098303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.098578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.099798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:46.100101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.101447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.109621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 556/1090] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:18:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:46.111015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.111656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.111937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.112211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.115751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.123073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:18:46.126454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:18:46.128059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:46.129103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:46.130053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 535/1091] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Aug 6 14:18:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:46.135709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.220976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a259de-f7ca-44a5-bce6-36e40116fca6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.235787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fd20-3487-4120-9905-695aec5ff749 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.250736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.255185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.255828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.256111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.256388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] PATH_INFO: `/v3/users/f2a74b17c62545659dff3dcc0befe61e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.257579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] RBAC: Authorizing `identity:get_user(user_id=f2a74b17c62545659dff3dcc0befe61e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.259025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.260643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 557/1092] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/users/f2a74b17c62545659dff3dcc0befe61e => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:46.266243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.271523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.272150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.272429 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.272703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.273409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:46.273712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.275101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.283274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 536/1093] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:46.289353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.343300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.366475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27076e6-3325-4fd1-a9ce-e38112cdea0e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.385390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f45393-c274-416b-85eb-860be366c33f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.399646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.400262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.400545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.400819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.419896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.420528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.420807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.421080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.432235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.433825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:46.450078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:46.451082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:46.452059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:46.452059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:46.452059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:46.452059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:46.452059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 537/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:46.457654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.468427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.469887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.488860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.544255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.558331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8184d-1110-4b0a-aa97-1f0abc87a78d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.568335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 558/1095] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 14:18:46.573790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.592430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.593039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.593315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.593585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.594287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:46.594585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.595974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.608218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 538/1096] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:46.613594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.670065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfd82b-60d7-4d0c-b205-4aa6fce3e558 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.697714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.704273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.704912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.705189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.705466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.706627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] RBAC: Authorizing `identity:get_project(project_id=ff54925983c24deeaf71b79a691b9bc1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.708080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.710245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 559/1097] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:46.710381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfc98d-43b6-4dc5-a432-790c222db2c7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.716158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.746035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.746591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.746893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.747186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.755717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.757148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.768644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:18:46.769310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:46.770264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 14:18:46.771935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:46.772476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 539/1098] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:18:46.778316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:46.828662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6911f-aba5-4b74-a852-069792d3a8fd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.864749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.865365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.865643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.865913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.866694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=ff54925983c24deeaf71b79a691b9bc1, user.id=f2a74b17c62545659dff3dcc0befe61e) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:46.867039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.868452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.874280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 560/1099] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/role_assignments?user.id=f2a74b17c62545659dff3dcc0befe61e&scope.project.id=ff54925983c24deeaf71b79a691b9bc1&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:46.882572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ba14c-9547-4b90-b82b-8123f294bc84 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:46.918922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:46.919561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:46.919843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:46.920122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:46.920937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:46.921248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:46.922734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:46.942200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 540/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:46.948632 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:47.054966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f64ef2-56e3-473e-8c44-9f49b8317ad4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:47.065619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:47.091565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:47.092189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:47.092471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:47.092745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:47.159527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:47.160963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:47.259717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 561/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 14:18:47.265367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:47.273977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:47.363340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8357e-0738-45d5-bb80-e557d24c125d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:47.396277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:47.396910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:47.397188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:47.397462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:47.407543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:47.408972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:18:47.423189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:47.424142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:47.425306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:18:47.425306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:18:47.425306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:18:47.425306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:47.425306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 541/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:47.430844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:47.440840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:47.530713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93e15-0924-42ea-a296-8a259ee796cc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:47.564984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:47.565596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:47.565875 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:47.566148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:47.566831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:47.567164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:47.568546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:47.579937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 562/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:47.585657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:47.642888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:47.684161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aed4c-efc4-4165-b0f5-63eb889503ec None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:47.717072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:47.717705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:47.717983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:47.718255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:47.724745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:47.726172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:18:47.737237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:18:47.738111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:18:47.739030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 542/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:18:47.744280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:47.809248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:47.843060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc16b7-441e-43d6-a7ab-1a582b1d60d3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:47.878440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e72566-31f1-46cd-b530-7d05c5ab70c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:47.879089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72566-31f1-46cd-b530-7d05c5ab70c7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:47.879371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72566-31f1-46cd-b530-7d05c5ab70c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:47.879644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72566-31f1-46cd-b530-7d05c5ab70c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:47.880923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e72566-31f1-46cd-b530-7d05c5ab70c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:47.881227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e72566-31f1-46cd-b530-7d05c5ab70c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:47.882627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e72566-31f1-46cd-b530-7d05c5ab70c7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:47.894176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 563/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:47.900134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:47.996828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae9ac2-1902-4435-b52a-5793b7c29f21 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:48.013978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:48.030591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df197504-b26d-4c3f-89eb-e2860d68fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:48.031232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197504-b26d-4c3f-89eb-e2860d68fa86 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:48.031510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197504-b26d-4c3f-89eb-e2860d68fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:48.031782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197504-b26d-4c3f-89eb-e2860d68fa86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:48.032549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197504-b26d-4c3f-89eb-e2860d68fa86 None admin] RBAC: Adding query filter params (role.id=7e2a69597b9f47a598002ebca7e97820, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=075e9c0ef8fd406087d05a6ce36dc4e2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:48.032859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197504-b26d-4c3f-89eb-e2860d68fa86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:48.034272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197504-b26d-4c3f-89eb-e2860d68fa86 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:48.039956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 543/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:47 2022] GET /identity/v3/role_assignments?user.id=075e9c0ef8fd406087d05a6ce36dc4e2&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=7e2a69597b9f47a598002ebca7e97820 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:18:48.873682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6cf56-aa2b-487f-8b49-c82e99d4cc4b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:48.874157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6cf56-aa2b-487f-8b49-c82e99d4cc4b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:48.874247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6cf56-aa2b-487f-8b49-c82e99d4cc4b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:48.875645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 564/1107] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:48.880950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d3ae-6af0-483e-b4b9-7caf6ed86b57 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:48.881348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d3ae-6af0-483e-b4b9-7caf6ed86b57 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:48.881506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d3ae-6af0-483e-b4b9-7caf6ed86b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:48.899485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b874d3ae-6af0-483e-b4b9-7caf6ed86b57 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:48.900652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b874d3ae-6af0-483e-b4b9-7caf6ed86b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:48.964712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 544/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:48.989170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6cf56-aa2b-487f-8b49-c82e99d4cc4b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:49.087495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6cf56-aa2b-487f-8b49-c82e99d4cc4b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:49.112305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:49.120168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7b306-939e-4cfc-95d6-4647dd4bc384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:49.120795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7b306-939e-4cfc-95d6-4647dd4bc384 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:49.121075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7b306-939e-4cfc-95d6-4647dd4bc384 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:49.121348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7b306-939e-4cfc-95d6-4647dd4bc384 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:49.122479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7b306-939e-4cfc-95d6-4647dd4bc384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:49.123933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7b306-939e-4cfc-95d6-4647dd4bc384 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:49.125501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 565/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:49.131594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874d3ae-6af0-483e-b4b9-7caf6ed86b57 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:49.225800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874d3ae-6af0-483e-b4b9-7caf6ed86b57 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:49.238116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:49.258782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d574c-eb1e-478a-a8c3-d4c66f8489cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:49.259418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d574c-eb1e-478a-a8c3-d4c66f8489cf None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:49.259696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d574c-eb1e-478a-a8c3-d4c66f8489cf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:49.259971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d574c-eb1e-478a-a8c3-d4c66f8489cf None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:49.260644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d574c-eb1e-478a-a8c3-d4c66f8489cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:49.262059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d574c-eb1e-478a-a8c3-d4c66f8489cf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:49.282646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 545/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:18:49 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:18:51.282340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96fbe0-8a2b-4dd4-8339-74c7e6e9344b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:51.282896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96fbe0-8a2b-4dd4-8339-74c7e6e9344b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:51.282896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96fbe0-8a2b-4dd4-8339-74c7e6e9344b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:51.284267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 566/1111] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:51.289966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ccc09-8ae7-4eb7-8118-d3030e66a4ab None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:51.290362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ccc09-8ae7-4eb7-8118-d3030e66a4ab None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:51.290520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ccc09-8ae7-4eb7-8118-d3030e66a4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:51.308861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c79ccc09-8ae7-4eb7-8118-d3030e66a4ab None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:51.310040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c79ccc09-8ae7-4eb7-8118-d3030e66a4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:51.374857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 546/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:51.397806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96fbe0-8a2b-4dd4-8339-74c7e6e9344b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:51.497076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96fbe0-8a2b-4dd4-8339-74c7e6e9344b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:51.530036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:51.530659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:51.530963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:51.531237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] PATH_INFO: `/v3/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:51.532700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] RBAC: Authorizing `identity:get_role(role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:51.533997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:51.535361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 567/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:51 2022] GET /identity/v3/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:51.541687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ccc09-8ae7-4eb7-8118-d3030e66a4ab None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:51.632179 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:51.635912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ccc09-8ae7-4eb7-8118-d3030e66a4ab None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:51.669768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:51.670402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:51.670676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:51.670975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] PATH_INFO: `/v3/users/66bc9a02002c4372a0130fea30f189da` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:51.683583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] RBAC: Authorizing `identity:get_user(user_id=66bc9a02002c4372a0130fea30f189da)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:51.684993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:51.686578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 547/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:51 2022] GET /identity/v3/users/66bc9a02002c4372a0130fea30f189da => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:51.692257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:51.752092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:51.788588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0dd36-1b9c-499d-b54b-0a8a65f507a8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:51.820592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:51.821192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:51.821468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:51.821738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:51.888048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:51.889473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:51.946929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:51.983877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 568/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:51.989857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:52.085194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c788f-31cb-4a62-94b5-12c1aeb4fef6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:52.116697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08146fe-a460-4933-a6d5-a3bdb551d22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:52.117321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08146fe-a460-4933-a6d5-a3bdb551d22f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:52.117597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08146fe-a460-4933-a6d5-a3bdb551d22f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:52.117866 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08146fe-a460-4933-a6d5-a3bdb551d22f None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:52.119023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08146fe-a460-4933-a6d5-a3bdb551d22f None admin] RBAC: Authorizing `identity:get_project(project_id=ff54925983c24deeaf71b79a691b9bc1)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:52.120442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08146fe-a460-4933-a6d5-a3bdb551d22f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:52.122050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 548/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:18:51 2022] GET /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:52.127835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:52.222439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f882122-b157-499b-98d7-72898ae84d29 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:52.254308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c14e7-db65-45b9-8cdd-607639da26c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:52.255440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c14e7-db65-45b9-8cdd-607639da26c8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:52.255721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c14e7-db65-45b9-8cdd-607639da26c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:52.255994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c14e7-db65-45b9-8cdd-607639da26c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:52.256761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c14e7-db65-45b9-8cdd-607639da26c8 None admin] RBAC: Adding query filter params (role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=ff54925983c24deeaf71b79a691b9bc1, user.id=66bc9a02002c4372a0130fea30f189da) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:52.257066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c14e7-db65-45b9-8cdd-607639da26c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:52.258451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c14e7-db65-45b9-8cdd-607639da26c8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:52.263662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 569/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:52 2022] GET /identity/v3/role_assignments?user.id=66bc9a02002c4372a0130fea30f189da&scope.project.id=ff54925983c24deeaf71b79a691b9bc1&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:54.196930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3644c-de34-4ac2-a0f0-03751b1d2e3b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:54.197462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3644c-de34-4ac2-a0f0-03751b1d2e3b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:54.197462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3644c-de34-4ac2-a0f0-03751b1d2e3b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:54.198820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 549/1118] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:54.203818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ca5f4-c549-4909-918e-74ad9baf5e5a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:54.204197 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ca5f4-c549-4909-918e-74ad9baf5e5a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:54.204369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ca5f4-c549-4909-918e-74ad9baf5e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:54.221948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-319ca5f4-c549-4909-918e-74ad9baf5e5a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:54.223156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-319ca5f4-c549-4909-918e-74ad9baf5e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:54.246049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:54.288596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 570/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:54.311048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3644c-de34-4ac2-a0f0-03751b1d2e3b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:54.335025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:54.407619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3644c-de34-4ac2-a0f0-03751b1d2e3b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:54.440776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:54.441408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:54.441691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:54.441964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] PATH_INFO: `/v3/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:54.443445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] RBAC: Authorizing `identity:get_role(role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:54.444808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:54.446134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 550/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:54 2022] GET /identity/v3/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:54.452101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ca5f4-c549-4909-918e-74ad9baf5e5a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:54.547406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ca5f4-c549-4909-918e-74ad9baf5e5a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:54.579580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:54.580203 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:54.580483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:54.580757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] PATH_INFO: `/v3/users/66bc9a02002c4372a0130fea30f189da` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:54.581896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] RBAC: Authorizing `identity:get_user(user_id=66bc9a02002c4372a0130fea30f189da)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:54.583979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:54.585557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 571/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:54 2022] GET /identity/v3/users/66bc9a02002c4372a0130fea30f189da => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:54.591157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:54.679230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:54.688220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640121bc-baba-43ea-bb82-de88ea9e25d9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:54.719953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522972f-c288-42e6-bad9-7b262d3eed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:54.720521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522972f-c288-42e6-bad9-7b262d3eed26 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:54.720794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522972f-c288-42e6-bad9-7b262d3eed26 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:54.721063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522972f-c288-42e6-bad9-7b262d3eed26 None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:54.722192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522972f-c288-42e6-bad9-7b262d3eed26 None admin] RBAC: Authorizing `identity:get_project(project_id=ff54925983c24deeaf71b79a691b9bc1)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:54.724251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522972f-c288-42e6-bad9-7b262d3eed26 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:54.725876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 551/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:18:54 2022] GET /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:54.731768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:54.757855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:54.827531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2806f-d4a0-4212-b316-1b1bda7bc042 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:54.860650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f3b76-71ee-4285-972d-e92f29d16384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:54.861339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f3b76-71ee-4285-972d-e92f29d16384 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:54.861618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f3b76-71ee-4285-972d-e92f29d16384 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:54.861891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f3b76-71ee-4285-972d-e92f29d16384 None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1/users/66bc9a02002c4372a0130fea30f189da/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:54.864039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f3b76-71ee-4285-972d-e92f29d16384 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff54925983c24deeaf71b79a691b9bc1, user_id=66bc9a02002c4372a0130fea30f189da, role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:54.865446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f3b76-71ee-4285-972d-e92f29d16384 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:54.878008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 572/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:18:54 2022] PUT /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1/users/66bc9a02002c4372a0130fea30f189da/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:18:56.908633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f32567-121c-4632-bd84-3bc6a06ba411 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:56.909154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f32567-121c-4632-bd84-3bc6a06ba411 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:56.909154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f32567-121c-4632-bd84-3bc6a06ba411 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:56.910526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 552/1124] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:56.915555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66efb7-0985-48d1-ad33-214f40d630a4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:56.915971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66efb7-0985-48d1-ad33-214f40d630a4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:56.916109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66efb7-0985-48d1-ad33-214f40d630a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:56.934422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cb66efb7-0985-48d1-ad33-214f40d630a4 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:56.935607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cb66efb7-0985-48d1-ad33-214f40d630a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:57.000455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 573/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:57.023666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f32567-121c-4632-bd84-3bc6a06ba411 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:57.121966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f32567-121c-4632-bd84-3bc6a06ba411 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:57.155175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bd117-0735-488f-8487-458496240480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:57.155810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bd117-0735-488f-8487-458496240480 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:57.156087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bd117-0735-488f-8487-458496240480 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:57.156358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bd117-0735-488f-8487-458496240480 None admin] PATH_INFO: `/v3/roles/2657e8501e1e4f63a0a1a261ff8d89be` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:57.157780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bd117-0735-488f-8487-458496240480 None admin] RBAC: Authorizing `identity:get_role(role_id=2657e8501e1e4f63a0a1a261ff8d89be)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:57.159176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bd117-0735-488f-8487-458496240480 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:57.160530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 553/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:57 2022] GET /identity/v3/roles/2657e8501e1e4f63a0a1a261ff8d89be => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:57.167002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66efb7-0985-48d1-ad33-214f40d630a4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:57.184706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:57.264678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66efb7-0985-48d1-ad33-214f40d630a4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:57.297197 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:57.297830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:57.298114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:57.298389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] PATH_INFO: `/v3/users/66bc9a02002c4372a0130fea30f189da` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:57.299542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] RBAC: Authorizing `identity:get_user(user_id=66bc9a02002c4372a0130fea30f189da)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:57.301211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:57.303433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 574/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:18:57 2022] GET /identity/v3/users/66bc9a02002c4372a0130fea30f189da => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:57.309200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bd117-0735-488f-8487-458496240480 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:57.336763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:57.406719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bd117-0735-488f-8487-458496240480 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:57.440149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:57.440770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:57.441048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:57.441323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:57.506954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:57.508409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:57.534115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:57.603516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 554/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:18:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:18:57.609456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:57.690645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:57.706177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11cf80-59af-42e6-b99d-1d29b0bb7e62 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:57.740177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91106eff-d31a-4938-a0c5-fc417b853b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:57.740810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91106eff-d31a-4938-a0c5-fc417b853b60 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:57.741086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91106eff-d31a-4938-a0c5-fc417b853b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:57.741357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91106eff-d31a-4938-a0c5-fc417b853b60 None admin] PATH_INFO: `/v3/projects/ff54925983c24deeaf71b79a691b9bc1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:57.742486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91106eff-d31a-4938-a0c5-fc417b853b60 None admin] RBAC: Authorizing `identity:get_project(project_id=ff54925983c24deeaf71b79a691b9bc1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:57.743931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91106eff-d31a-4938-a0c5-fc417b853b60 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:57.745579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 575/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:18:57 2022] GET /identity/v3/projects/ff54925983c24deeaf71b79a691b9bc1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:57.751887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:18:57.846683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7608513-6158-4ef0-a35f-05628f4f6371 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:18:57.872605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:18:57.881005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f78d-ad75-4aff-9708-fef0e3a7c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:18:57.881610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f78d-ad75-4aff-9708-fef0e3a7c47a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:57.881887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f78d-ad75-4aff-9708-fef0e3a7c47a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:57.882154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f78d-ad75-4aff-9708-fef0e3a7c47a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:57.882951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f78d-ad75-4aff-9708-fef0e3a7c47a None admin] RBAC: Adding query filter params (role.id=2657e8501e1e4f63a0a1a261ff8d89be, scope.project.id=ff54925983c24deeaf71b79a691b9bc1, user.id=66bc9a02002c4372a0130fea30f189da) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:18:57.883259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f78d-ad75-4aff-9708-fef0e3a7c47a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:18:57.884679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f78d-ad75-4aff-9708-fef0e3a7c47a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:18:57.890295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 555/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:18:57 2022] GET /identity/v3/role_assignments?user.id=66bc9a02002c4372a0130fea30f189da&scope.project.id=ff54925983c24deeaf71b79a691b9bc1&role.id=2657e8501e1e4f63a0a1a261ff8d89be => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:18:59.870371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8320-0f61-4a74-834d-f58b819cc454 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:59.870829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8320-0f61-4a74-834d-f58b819cc454 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:59.870964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8320-0f61-4a74-834d-f58b819cc454 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:59.872280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 576/1131] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:18:59.877337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a571-9d04-44fe-9937-becc10a96671 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:18:59.877738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a571-9d04-44fe-9937-becc10a96671 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:18:59.877887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a571-9d04-44fe-9937-becc10a96671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:18:59.896093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ff86a571-9d04-44fe-9937-becc10a96671 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:18:59.897256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ff86a571-9d04-44fe-9937-becc10a96671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:18:59.961911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 556/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:18:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:18:59.984352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8320-0f61-4a74-834d-f58b819cc454 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:00.081025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8320-0f61-4a74-834d-f58b819cc454 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:00.112572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450d7d3-bdcd-4e85-b346-65ca08f2b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:00.113134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d7d3-bdcd-4e85-b346-65ca08f2b96c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:00.113412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d7d3-bdcd-4e85-b346-65ca08f2b96c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:00.113682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d7d3-bdcd-4e85-b346-65ca08f2b96c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:00.115335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450d7d3-bdcd-4e85-b346-65ca08f2b96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:00.116727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450d7d3-bdcd-4e85-b346-65ca08f2b96c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:00.118258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 577/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:18:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:00.123907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a571-9d04-44fe-9937-becc10a96671 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:00.217778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a571-9d04-44fe-9937-becc10a96671 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:00.239951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:00.251401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a66691-a139-460c-be51-7c275b06f2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:00.252004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a66691-a139-460c-be51-7c275b06f2d7 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:00.252284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a66691-a139-460c-be51-7c275b06f2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:00.252555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a66691-a139-460c-be51-7c275b06f2d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:00.253236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a66691-a139-460c-be51-7c275b06f2d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:00.254674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a66691-a139-460c-be51-7c275b06f2d7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:00.281255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 557/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:19:00 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:19:02.262610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbd012-d392-4126-abeb-9cc66760c303 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:02.263204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbd012-d392-4126-abeb-9cc66760c303 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:02.263204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbd012-d392-4126-abeb-9cc66760c303 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:02.264534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 578/1135] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:02.269536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd01d2-ce91-40ca-9636-656e8f310721 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:02.269975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd01d2-ce91-40ca-9636-656e8f310721 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:02.270596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd01d2-ce91-40ca-9636-656e8f310721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:02.288793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1dbd01d2-ce91-40ca-9636-656e8f310721 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:02.289974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1dbd01d2-ce91-40ca-9636-656e8f310721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:02.353700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 558/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:02.376718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbd012-d392-4126-abeb-9cc66760c303 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:02.398415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:02.484264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbd012-d392-4126-abeb-9cc66760c303 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:02.516690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d53bca-9465-468d-9bab-3b882858fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:02.517261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d53bca-9465-468d-9bab-3b882858fb20 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:02.517536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d53bca-9465-468d-9bab-3b882858fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:02.517803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d53bca-9465-468d-9bab-3b882858fb20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:02.518946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d53bca-9465-468d-9bab-3b882858fb20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:02.520951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d53bca-9465-468d-9bab-3b882858fb20 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:02.522510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 579/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:19:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:02.528527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd01d2-ce91-40ca-9636-656e8f310721 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:02.623030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd01d2-ce91-40ca-9636-656e8f310721 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:02.625253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:02.657295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c061-2f3f-4a4d-8a72-5d4dfdb4afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:02.657904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c061-2f3f-4a4d-8a72-5d4dfdb4afde None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:02.658184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c061-2f3f-4a4d-8a72-5d4dfdb4afde None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:02.658457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c061-2f3f-4a4d-8a72-5d4dfdb4afde None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:02.659178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0c061-2f3f-4a4d-8a72-5d4dfdb4afde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:02.660607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0c061-2f3f-4a4d-8a72-5d4dfdb4afde None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:02.680549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 559/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:19:02 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:19:04.674098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebeea3-7722-42fa-a410-2e83461a6825 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:04.674622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebeea3-7722-42fa-a410-2e83461a6825 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:04.674622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebeea3-7722-42fa-a410-2e83461a6825 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:04.676260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 580/1139] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:04.681567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb000-470b-4851-bc42-df283177f7c5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:04.681951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb000-470b-4851-bc42-df283177f7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:04.682128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb000-470b-4851-bc42-df283177f7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:04.700248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ba2cb000-470b-4851-bc42-df283177f7c5 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:04.701434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ba2cb000-470b-4851-bc42-df283177f7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:04.765232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 560/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:04.797036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebeea3-7722-42fa-a410-2e83461a6825 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:04.884774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:04.897760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebeea3-7722-42fa-a410-2e83461a6825 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:04.929746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:04.930317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:04.930593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:04.930862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:04.940475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:19:04.945817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:19:04.946983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:19:04.948033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 581/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:19:04.953138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cb000-470b-4851-bc42-df283177f7c5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:05.041208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:05.052030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cb000-470b-4851-bc42-df283177f7c5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:05.084274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:05.084878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:05.085154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:05.085424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:05.086119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:05.086420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:05.087795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:05.095436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 561/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:05.101145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:05.209411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a11413-f511-442e-a9a7-eb378cd838c6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:05.241480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:05.242100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:05.242377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:05.242647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] PATH_INFO: `/v3/users/3f5df2dac8e14be99c1d56fc5b7c9882` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:05.255601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] RBAC: Authorizing `identity:get_user(user_id=3f5df2dac8e14be99c1d56fc5b7c9882)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:05.257006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:05.258580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 582/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:19:05 2022] GET /identity/v3/users/3f5df2dac8e14be99c1d56fc5b7c9882 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:05.264391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:05.359630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d06cf-c6c7-4084-aa2a-249e3935413b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:05.392046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:05.392654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:05.392935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:05.393211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:05.413667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:05.458569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:05.460096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:05.555271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 562/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:19:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:19:05.561118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:05.569591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:05.655682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fa4cf-b2ca-4871-8f3e-7aef10ab342f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:05.688626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e62f3-886c-46f6-912d-1b731d47b6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:05.689252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e62f3-886c-46f6-912d-1b731d47b6ca None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:05.689526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e62f3-886c-46f6-912d-1b731d47b6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:05.689794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e62f3-886c-46f6-912d-1b731d47b6ca None admin] PATH_INFO: `/v3/projects/81bdba5593134c7a8f4c7f74046aee6e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:05.690942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e62f3-886c-46f6-912d-1b731d47b6ca None admin] RBAC: Authorizing `identity:get_project(project_id=81bdba5593134c7a8f4c7f74046aee6e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:05.692383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e62f3-886c-46f6-912d-1b731d47b6ca None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:05.693999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 583/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:19:05 2022] GET /identity/v3/projects/81bdba5593134c7a8f4c7f74046aee6e => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:05.700057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:05.750975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:05.795247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248994c-2a83-42a6-9818-18e382a9ca1a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:05.827050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348fec1-2ad8-475c-adb0-9adcc560fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:05.827650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348fec1-2ad8-475c-adb0-9adcc560fedf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:05.827927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348fec1-2ad8-475c-adb0-9adcc560fedf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:05.828197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348fec1-2ad8-475c-adb0-9adcc560fedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:05.828958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348fec1-2ad8-475c-adb0-9adcc560fedf None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=81bdba5593134c7a8f4c7f74046aee6e, user.id=3f5df2dac8e14be99c1d56fc5b7c9882) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:05.829263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348fec1-2ad8-475c-adb0-9adcc560fedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:05.830681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348fec1-2ad8-475c-adb0-9adcc560fedf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:05.836334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 563/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:19:05 2022] GET /identity/v3/role_assignments?user.id=3f5df2dac8e14be99c1d56fc5b7c9882&scope.project.id=81bdba5593134c7a8f4c7f74046aee6e&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:07.799899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825e601-b86e-4a5d-aa4f-16e41ff1add2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:07.800676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825e601-b86e-4a5d-aa4f-16e41ff1add2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:07.800910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825e601-b86e-4a5d-aa4f-16e41ff1add2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:07.803082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 584/1147] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:07.808155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573095b-63fe-4609-adab-12c9fc98f5a3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:07.808549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573095b-63fe-4609-adab-12c9fc98f5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:07.808704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573095b-63fe-4609-adab-12c9fc98f5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:07.826651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5573095b-63fe-4609-adab-12c9fc98f5a3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:07.827873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5573095b-63fe-4609-adab-12c9fc98f5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:07.892004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 564/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:19:07.914189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825e601-b86e-4a5d-aa4f-16e41ff1add2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:07.986771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:08.011849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825e601-b86e-4a5d-aa4f-16e41ff1add2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:08.044212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:08.044829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:08.045105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:08.045375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:08.055951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:19:08.060697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:19:08.061792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:19:08.062707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 585/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:19:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:19:08.068221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573095b-63fe-4609-adab-12c9fc98f5a3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:08.111802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:08.163151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573095b-63fe-4609-adab-12c9fc98f5a3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:08.196134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:08.196749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:08.197031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:08.197305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:08.198005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:08.198312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:08.199711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:08.207477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 565/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:19:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:08.213347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:08.309513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7e492-6fb2-4abf-a133-8594164c4cd4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:08.341899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:08.342519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:08.342801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:08.343102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] PATH_INFO: `/v3/users/3f5df2dac8e14be99c1d56fc5b7c9882` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:08.344243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] RBAC: Authorizing `identity:get_user(user_id=3f5df2dac8e14be99c1d56fc5b7c9882)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:08.345652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:08.347246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 586/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:19:08 2022] GET /identity/v3/users/3f5df2dac8e14be99c1d56fc5b7c9882 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:08.352862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:08.448438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84ed98-a612-40da-8cff-373a10add4c0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:08.456811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:08.481111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7920b-f1df-4e1e-b6d4-62c9f31bf232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:08.481732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7920b-f1df-4e1e-b6d4-62c9f31bf232 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:08.482007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7920b-f1df-4e1e-b6d4-62c9f31bf232 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:08.482274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7920b-f1df-4e1e-b6d4-62c9f31bf232 None admin] PATH_INFO: `/v3/projects/81bdba5593134c7a8f4c7f74046aee6e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:08.483416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7920b-f1df-4e1e-b6d4-62c9f31bf232 None admin] RBAC: Authorizing `identity:get_project(project_id=81bdba5593134c7a8f4c7f74046aee6e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:08.484823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7920b-f1df-4e1e-b6d4-62c9f31bf232 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:08.486448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 566/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:19:08 2022] GET /identity/v3/projects/81bdba5593134c7a8f4c7f74046aee6e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:08.492899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:08.495699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:08.589364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262daac9-abfa-4e71-a021-c8dd8d2d5f2d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:08.622638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a572428-1961-45f3-bb05-896a85d93e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:08.623315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a572428-1961-45f3-bb05-896a85d93e9b None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:08.623597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a572428-1961-45f3-bb05-896a85d93e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:08.623870 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a572428-1961-45f3-bb05-896a85d93e9b None admin] PATH_INFO: `/v3/projects/81bdba5593134c7a8f4c7f74046aee6e/users/3f5df2dac8e14be99c1d56fc5b7c9882/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:08.625782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a572428-1961-45f3-bb05-896a85d93e9b None admin] RBAC: Authorizing `identity:create_grant(project_id=81bdba5593134c7a8f4c7f74046aee6e, user_id=3f5df2dac8e14be99c1d56fc5b7c9882, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:08.627216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a572428-1961-45f3-bb05-896a85d93e9b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:08.639492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 587/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:19:08 2022] PUT /identity/v3/projects/81bdba5593134c7a8f4c7f74046aee6e/users/3f5df2dac8e14be99c1d56fc5b7c9882/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:19:10.615175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c36a-d1f7-40f5-870e-26c10203d8ac None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:10.615728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c36a-d1f7-40f5-870e-26c10203d8ac None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:10.615728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c36a-d1f7-40f5-870e-26c10203d8ac None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:10.617072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 567/1154] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:10.622110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04104e0c-5429-4491-8b50-5b4402556754 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:10.622498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04104e0c-5429-4491-8b50-5b4402556754 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:10.622654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04104e0c-5429-4491-8b50-5b4402556754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:10.640924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-04104e0c-5429-4491-8b50-5b4402556754 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:10.642102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04104e0c-5429-4491-8b50-5b4402556754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:10.697968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:10.710455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 588/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:10.733490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504c36a-d1f7-40f5-870e-26c10203d8ac None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:10.831612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504c36a-d1f7-40f5-870e-26c10203d8ac None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:10.864410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:10.865042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:10.865364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:10.865590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:10.875844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:19:10.881392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:19:10.882497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:19:10.883374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 568/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:19:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:19:10.888579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04104e0c-5429-4491-8b50-5b4402556754 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:10.983455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04104e0c-5429-4491-8b50-5b4402556754 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:11.015945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:11.016546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:11.016829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:11.017106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:11.017811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:11.018117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:11.019496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:11.027820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 589/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:19:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:11.033632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:11.068564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:11.129127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab086f5-e767-40f8-8329-0132c3c78f99 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:11.163298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:11.163932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:11.164209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:11.164482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] PATH_INFO: `/v3/users/3f5df2dac8e14be99c1d56fc5b7c9882` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:11.165632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] RBAC: Authorizing `identity:get_user(user_id=3f5df2dac8e14be99c1d56fc5b7c9882)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:11.167087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:11.168697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 569/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:19:11 2022] GET /identity/v3/users/3f5df2dac8e14be99c1d56fc5b7c9882 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:11.174482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:11.228015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:11.270774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad679de8-d087-479b-885c-e984edcf5bb1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:11.303011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:11.303613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:11.303889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:11.304156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:11.367754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:11.369180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:11.420193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:11.464001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 590/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:19:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:19:11.469989 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:11.580596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163082ce-8d86-4e0e-8b14-3f803cdee55a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:11.582937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:11.613164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c8dff-9d13-4c43-97f9-78e73f1ce3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:11.614389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8dff-9d13-4c43-97f9-78e73f1ce3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:11.614662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8dff-9d13-4c43-97f9-78e73f1ce3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:11.614962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8dff-9d13-4c43-97f9-78e73f1ce3e3 None admin] PATH_INFO: `/v3/projects/81bdba5593134c7a8f4c7f74046aee6e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:11.616099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c8dff-9d13-4c43-97f9-78e73f1ce3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=81bdba5593134c7a8f4c7f74046aee6e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:11.617512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c8dff-9d13-4c43-97f9-78e73f1ce3e3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:11.619217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 570/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:19:11 2022] GET /identity/v3/projects/81bdba5593134c7a8f4c7f74046aee6e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:11.625378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:11.719986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561e407-39e6-4ad9-bf93-a7b032c6f560 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:11.752706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd1c97-9072-4b9c-b7fb-a42ba26d8b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:11.753302 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1c97-9072-4b9c-b7fb-a42ba26d8b2e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:11.753578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1c97-9072-4b9c-b7fb-a42ba26d8b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:11.753848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1c97-9072-4b9c-b7fb-a42ba26d8b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:11.754640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd1c97-9072-4b9c-b7fb-a42ba26d8b2e None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=81bdba5593134c7a8f4c7f74046aee6e, user.id=3f5df2dac8e14be99c1d56fc5b7c9882) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:11.754970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd1c97-9072-4b9c-b7fb-a42ba26d8b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:11.756372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd1c97-9072-4b9c-b7fb-a42ba26d8b2e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:11.761905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 591/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:19:11 2022] GET /identity/v3/role_assignments?user.id=3f5df2dac8e14be99c1d56fc5b7c9882&scope.project.id=81bdba5593134c7a8f4c7f74046aee6e&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:13.760581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d097a1-394c-4092-a51b-e37c174a10a4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:13.761112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d097a1-394c-4092-a51b-e37c174a10a4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:13.761112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d097a1-394c-4092-a51b-e37c174a10a4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:13.762471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 571/1162] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:13.767763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec858e-3853-4711-96f7-09633a6f8049 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:13.768156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec858e-3853-4711-96f7-09633a6f8049 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:13.768305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec858e-3853-4711-96f7-09633a6f8049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:13.786402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-66ec858e-3853-4711-96f7-09633a6f8049 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:13.787608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-66ec858e-3853-4711-96f7-09633a6f8049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:13.800557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:13.854228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 592/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:13.876209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d097a1-394c-4092-a51b-e37c174a10a4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:13.973632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d097a1-394c-4092-a51b-e37c174a10a4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:14.006135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:14.006740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:14.007052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:14.007328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:14.046341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:14.071898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:14.073332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:14.167622 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 572/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:19:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:19:14.173505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec858e-3853-4711-96f7-09633a6f8049 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:14.267681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec858e-3853-4711-96f7-09633a6f8049 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:14.301115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede779-741d-43e1-a339-c9c7378eb983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:14.301690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede779-741d-43e1-a339-c9c7378eb983 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:14.301974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede779-741d-43e1-a339-c9c7378eb983 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:14.302250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede779-741d-43e1-a339-c9c7378eb983 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:14.308738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede779-741d-43e1-a339-c9c7378eb983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:14.310158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede779-741d-43e1-a339-c9c7378eb983 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a1ede779-741d-43e1-a339-c9c7378eb983 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:19:14.321435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:19:14.322653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:19:14.323750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 593/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:19:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:19:14.328434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:14.394196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:14.422811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e89b0-a085-4d10-9b87-a4b2c5843b3f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:14.455201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987c7be-1a93-40b9-9361-7e589a68a30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:14.455805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987c7be-1a93-40b9-9361-7e589a68a30d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:14.456081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987c7be-1a93-40b9-9361-7e589a68a30d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:14.456348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987c7be-1a93-40b9-9361-7e589a68a30d None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:14.457022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987c7be-1a93-40b9-9361-7e589a68a30d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:14.457921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987c7be-1a93-40b9-9361-7e589a68a30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:14.459304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987c7be-1a93-40b9-9361-7e589a68a30d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:14.466118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 573/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Aug 6 14:19:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:16.410939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b99775-e493-4378-87a8-34d364d1cb23 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:16.411490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b99775-e493-4378-87a8-34d364d1cb23 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:16.411490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b99775-e493-4378-87a8-34d364d1cb23 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:16.412843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 594/1167] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:16.417874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef46794-708a-42bf-9f7b-7a18865b781a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:16.418277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef46794-708a-42bf-9f7b-7a18865b781a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:16.418420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef46794-708a-42bf-9f7b-7a18865b781a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:16.436593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4ef46794-708a-42bf-9f7b-7a18865b781a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:16.437757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4ef46794-708a-42bf-9f7b-7a18865b781a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:16.500441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 574/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:19:16.522759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b99775-e493-4378-87a8-34d364d1cb23 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:16.533339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:16.621751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b99775-e493-4378-87a8-34d364d1cb23 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:16.654005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a3967-21d0-4a20-ac25-b69a0b01e848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:16.654606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3967-21d0-4a20-ac25-b69a0b01e848 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:16.654902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3967-21d0-4a20-ac25-b69a0b01e848 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:16.655179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3967-21d0-4a20-ac25-b69a0b01e848 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:16.655869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a3967-21d0-4a20-ac25-b69a0b01e848 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:16.657216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a3967-21d0-4a20-ac25-b69a0b01e848 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:16.672992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 595/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Sat Aug 6 14:19:16 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:19:18.711161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1dbac-9768-43b0-98f9-55e298124442 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:18.711691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1dbac-9768-43b0-98f9-55e298124442 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:18.711691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1dbac-9768-43b0-98f9-55e298124442 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:18.713091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 575/1170] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:18.718327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bb523-d926-43bf-b475-250596443f49 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:18.718736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bb523-d926-43bf-b475-250596443f49 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:18.718889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bb523-d926-43bf-b475-250596443f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:18.736957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ae3bb523-d926-43bf-b475-250596443f49 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:18.738141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ae3bb523-d926-43bf-b475-250596443f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:18.797183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:18.805120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 596/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:19:18.827810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1dbac-9768-43b0-98f9-55e298124442 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:18.903198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:18.938131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1dbac-9768-43b0-98f9-55e298124442 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:18.970822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4a43-e9b7-465b-bdba-7273d78a794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:18.971456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4a43-e9b7-465b-bdba-7273d78a794c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:18.971759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4a43-e9b7-465b-bdba-7273d78a794c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:18.972031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4a43-e9b7-465b-bdba-7273d78a794c None admin] PATH_INFO: `/v3/domains/3d59d90a79734be39dde47c6fbcd47bd` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:18.982054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc4a43-e9b7-465b-bdba-7273d78a794c None admin] RBAC: Authorizing `identity:get_domain(domain_id=3d59d90a79734be39dde47c6fbcd47bd)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:18.983543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc4a43-e9b7-465b-bdba-7273d78a794c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:18.985145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 576/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Sat Aug 6 14:19:18 2022] GET /identity/v3/domains/3d59d90a79734be39dde47c6fbcd47bd => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:18.991030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bb523-d926-43bf-b475-250596443f49 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:19.085931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bb523-d926-43bf-b475-250596443f49 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:19.117755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9951304-a256-4687-b821-0803d3e6c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:19.118358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9951304-a256-4687-b821-0803d3e6c2ff None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:19.118638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9951304-a256-4687-b821-0803d3e6c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:19.118936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9951304-a256-4687-b821-0803d3e6c2ff None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:19.119606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9951304-a256-4687-b821-0803d3e6c2ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:19.121019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9951304-a256-4687-b821-0803d3e6c2ff None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:19.146966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 597/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:19:18 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:19:21.163706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3759e45-246d-49b1-8316-ca392aeb6c65 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:21.164227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3759e45-246d-49b1-8316-ca392aeb6c65 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:21.164227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3759e45-246d-49b1-8316-ca392aeb6c65 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:21.165590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 577/1174] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:21.170587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9218a-d978-45e3-8350-f8e190d5aafe None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:21.170984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9218a-d978-45e3-8350-f8e190d5aafe None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:21.171164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9218a-d978-45e3-8350-f8e190d5aafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:21.189636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-95f9218a-d978-45e3-8350-f8e190d5aafe None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:21.190807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-95f9218a-d978-45e3-8350-f8e190d5aafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:21.234676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:21.255967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 598/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:21.279060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3759e45-246d-49b1-8316-ca392aeb6c65 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:21.376630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3759e45-246d-49b1-8316-ca392aeb6c65 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:21.400470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:21.409360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbb667-9335-453e-a8dc-4e09e44a02b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:21.409989 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbb667-9335-453e-a8dc-4e09e44a02b8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:21.410760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbb667-9335-453e-a8dc-4e09e44a02b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:21.411066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbb667-9335-453e-a8dc-4e09e44a02b8 None admin] PATH_INFO: `/v3/domains/3d59d90a79734be39dde47c6fbcd47bd` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:21.412207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbb667-9335-453e-a8dc-4e09e44a02b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3d59d90a79734be39dde47c6fbcd47bd)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:21.413630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbb667-9335-453e-a8dc-4e09e44a02b8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:21.415279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 578/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Sat Aug 6 14:19:21 2022] GET /identity/v3/domains/3d59d90a79734be39dde47c6fbcd47bd => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:21.420778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9218a-d978-45e3-8350-f8e190d5aafe None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:21.516396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9218a-d978-45e3-8350-f8e190d5aafe None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:21.549274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d393a-cf82-4161-8419-cf32160e2afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:21.549869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d393a-cf82-4161-8419-cf32160e2afb None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:21.550413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d393a-cf82-4161-8419-cf32160e2afb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:21.550710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d393a-cf82-4161-8419-cf32160e2afb None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:21.551424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d393a-cf82-4161-8419-cf32160e2afb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:21.552839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d393a-cf82-4161-8419-cf32160e2afb None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:21.573649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 599/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:19:21 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:19:23.577409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a9330-c3ae-45cb-8c50-51d0a3fbde5a None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:23.577936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a9330-c3ae-45cb-8c50-51d0a3fbde5a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:23.577936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a9330-c3ae-45cb-8c50-51d0a3fbde5a None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:23.579333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 579/1178] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:23.584438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339f2d-e949-49b9-b532-1fbbd29b7c65 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:23.584804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339f2d-e949-49b9-b532-1fbbd29b7c65 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:23.584988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339f2d-e949-49b9-b532-1fbbd29b7c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:23.602914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-07339f2d-e949-49b9-b532-1fbbd29b7c65 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:23.604082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-07339f2d-e949-49b9-b532-1fbbd29b7c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:23.631646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:23.668093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 600/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:23.690973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a9330-c3ae-45cb-8c50-51d0a3fbde5a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:23.788012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a9330-c3ae-45cb-8c50-51d0a3fbde5a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:23.825322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:23.825947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:23.826224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:23.826498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:23.836773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:19:23.842945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:19:23.843883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:19:23.844732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 580/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:19:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:19:23.849296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07339f2d-e949-49b9-b532-1fbbd29b7c65 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:23.943357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07339f2d-e949-49b9-b532-1fbbd29b7c65 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:23.976249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:23.976856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:23.977136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:23.977410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:23.978110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:23.978413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:23.979784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:23.987511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 601/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:19:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:23.993390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:24.047131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:24.089874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dea6fb-b603-4601-8b19-cd710d1b716e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:24.122400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:24.123049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:24.123330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:24.123603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] PATH_INFO: `/v3/users/a11feb327e134cfa81764c35dd905a37` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:24.135991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] RBAC: Authorizing `identity:get_user(user_id=a11feb327e134cfa81764c35dd905a37)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:24.137426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:24.139041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 581/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:19:23 2022] GET /identity/v3/users/a11feb327e134cfa81764c35dd905a37 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:24.145177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:24.172953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:24.241253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdd269-c479-4443-9642-edab33e41276 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:24.273830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:24.274423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:24.274705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:24.275004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:24.338573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:24.340061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:24.364849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:24.432981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 602/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:19:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:19:24.438744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:24.533904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e60910-0f67-403a-a1ab-94042c263f71 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:24.546380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:24.568045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ace3da-12f4-446c-b4b8-4ba953490b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:24.568675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace3da-12f4-446c-b4b8-4ba953490b12 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:24.568956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace3da-12f4-446c-b4b8-4ba953490b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:24.569230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace3da-12f4-446c-b4b8-4ba953490b12 None admin] PATH_INFO: `/v3/projects/a4369623e88e47428b14f0f10398a29f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:24.570369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ace3da-12f4-446c-b4b8-4ba953490b12 None admin] RBAC: Authorizing `identity:get_project(project_id=a4369623e88e47428b14f0f10398a29f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:24.571814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ace3da-12f4-446c-b4b8-4ba953490b12 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:24.573462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 582/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:19:24 2022] GET /identity/v3/projects/a4369623e88e47428b14f0f10398a29f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:24.579421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:24.672881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d5e06-7b20-4ebb-ba57-d9414411eb37 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:24.699592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:24.707436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e7d2f-a325-40dc-82aa-e4c3f3e47e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:24.708040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7d2f-a325-40dc-82aa-e4c3f3e47e99 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:24.708317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7d2f-a325-40dc-82aa-e4c3f3e47e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:24.708587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7d2f-a325-40dc-82aa-e4c3f3e47e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:24.709348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7d2f-a325-40dc-82aa-e4c3f3e47e99 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=a4369623e88e47428b14f0f10398a29f, user.id=a11feb327e134cfa81764c35dd905a37) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:24.709650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7d2f-a325-40dc-82aa-e4c3f3e47e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:24.711060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7d2f-a325-40dc-82aa-e4c3f3e47e99 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:24.716175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 603/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:19:24 2022] GET /identity/v3/role_assignments?user.id=a11feb327e134cfa81764c35dd905a37&scope.project.id=a4369623e88e47428b14f0f10398a29f&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:26.663379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15810621-f4f4-4535-886e-edb1e4ef08a5 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:26.663929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15810621-f4f4-4535-886e-edb1e4ef08a5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:26.663929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15810621-f4f4-4535-886e-edb1e4ef08a5 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:26.665252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 583/1186] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:19:26.670341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5d5c2-e772-4b6b-9882-e21daea82d21 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:26.670726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5d5c2-e772-4b6b-9882-e21daea82d21 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:26.670917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5d5c2-e772-4b6b-9882-e21daea82d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:26.689563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-05d5d5c2-e772-4b6b-9882-e21daea82d21 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:26.690751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-05d5d5c2-e772-4b6b-9882-e21daea82d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:26.755091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 604/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:26.777043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15810621-f4f4-4535-886e-edb1e4ef08a5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:26.877302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15810621-f4f4-4535-886e-edb1e4ef08a5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:26.914623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:26.915207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:26.915483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:26.915754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:26.927556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:19:26.932514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:19:26.933650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:19:26.934518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 584/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:19:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:19:26.940115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5d5c2-e772-4b6b-9882-e21daea82d21 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:27.034768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5d5c2-e772-4b6b-9882-e21daea82d21 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:27.057008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:27.068616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7756135-1777-4afa-a637-e71649365078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:27.069226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7756135-1777-4afa-a637-e71649365078 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:27.069506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7756135-1777-4afa-a637-e71649365078 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:27.069777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7756135-1777-4afa-a637-e71649365078 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:27.070480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7756135-1777-4afa-a637-e71649365078 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:27.070784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7756135-1777-4afa-a637-e71649365078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:27.072160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7756135-1777-4afa-a637-e71649365078 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:27.079889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 605/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:19:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:27.086126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:27.126849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:27.182951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2aa5f-a215-4a23-83dc-1dfa58fc24eb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:27.215721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:27.216344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:27.216621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:27.216895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] PATH_INFO: `/v3/users/a11feb327e134cfa81764c35dd905a37` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:27.218051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] RBAC: Authorizing `identity:get_user(user_id=a11feb327e134cfa81764c35dd905a37)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:27.219508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:27.222215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 585/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:19:27 2022] GET /identity/v3/users/a11feb327e134cfa81764c35dd905a37 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:27.228138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7756135-1777-4afa-a637-e71649365078 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:27.332781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7756135-1777-4afa-a637-e71649365078 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:27.366399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2635e5-5ccd-438c-ad84-b5d4520cbd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:27.367059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2635e5-5ccd-438c-ad84-b5d4520cbd5f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:27.367340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2635e5-5ccd-438c-ad84-b5d4520cbd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:27.367609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2635e5-5ccd-438c-ad84-b5d4520cbd5f None admin] PATH_INFO: `/v3/projects/a4369623e88e47428b14f0f10398a29f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:27.368736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2635e5-5ccd-438c-ad84-b5d4520cbd5f None admin] RBAC: Authorizing `identity:get_project(project_id=a4369623e88e47428b14f0f10398a29f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:27.370131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2635e5-5ccd-438c-ad84-b5d4520cbd5f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:27.371806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 606/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:19:27 2022] GET /identity/v3/projects/a4369623e88e47428b14f0f10398a29f => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:27.377712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:27.473560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf94899-1cef-499b-bebf-f6f56be414ac None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:27.481814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:27.507099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1479f3-6566-4e55-be8a-ae18309ed1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:27.507756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1479f3-6566-4e55-be8a-ae18309ed1ac None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:27.508036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1479f3-6566-4e55-be8a-ae18309ed1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:27.508310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1479f3-6566-4e55-be8a-ae18309ed1ac None admin] PATH_INFO: `/v3/projects/a4369623e88e47428b14f0f10398a29f/users/a11feb327e134cfa81764c35dd905a37/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:27.510220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1479f3-6566-4e55-be8a-ae18309ed1ac None admin] RBAC: Authorizing `identity:create_grant(project_id=a4369623e88e47428b14f0f10398a29f, user_id=a11feb327e134cfa81764c35dd905a37, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:27.511675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1479f3-6566-4e55-be8a-ae18309ed1ac None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:27.524377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 586/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:19:27 2022] PUT /identity/v3/projects/a4369623e88e47428b14f0f10398a29f/users/a11feb327e134cfa81764c35dd905a37/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:19:29.495656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de3c43-108f-4648-8523-115230cbbc88 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:29.496280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de3c43-108f-4648-8523-115230cbbc88 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:29.496280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de3c43-108f-4648-8523-115230cbbc88 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:29.497540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 607/1193] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:19:29.502589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b93fc-c415-45cf-8f1c-7605d105a4a3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:29.503044 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b93fc-c415-45cf-8f1c-7605d105a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:29.503171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b93fc-c415-45cf-8f1c-7605d105a4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:29.521704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f92b93fc-c415-45cf-8f1c-7605d105a4a3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:19:29.522887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f92b93fc-c415-45cf-8f1c-7605d105a4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:19:29.586803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 587/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:19:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:19:29.609448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de3c43-108f-4648-8523-115230cbbc88 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:29.661383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:29.708094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de3c43-108f-4648-8523-115230cbbc88 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:29.740167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:29.740788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:29.741066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:29.741338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:29.751480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:19:29.756753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:19:29.757905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:19:29.759017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 608/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:19:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:19:29.763884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b93fc-c415-45cf-8f1c-7605d105a4a3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:29.856079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:29.859917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b93fc-c415-45cf-8f1c-7605d105a4a3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:29.894570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:29.895227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:29.895508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:29.895779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:29.896477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:29.896780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:29.898136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:29.905861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 588/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:19:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:29.911600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:30.007671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689909af-c4d8-4cb7-9997-432c2dfff066 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:30.019582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:30.039872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:30.040505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:30.040786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:30.041062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] PATH_INFO: `/v3/users/a11feb327e134cfa81764c35dd905a37` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:30.042207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] RBAC: Authorizing `identity:get_user(user_id=a11feb327e134cfa81764c35dd905a37)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:30.043644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:30.045230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 609/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Aug 6 14:19:29 2022] GET /identity/v3/users/a11feb327e134cfa81764c35dd905a37 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:30.051811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:30.147664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4678d3-1401-4391-b1d1-90d79b96c196 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:30.182365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:30.183024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:30.183360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:30.183582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:30.210657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:30.251033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:30.252490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:30.347893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 589/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:19:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:19:30.353658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:30.454580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55cbde-39ab-4a09-a498-88172c09ceb9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:30.488757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2d2b-9c86-4786-a7df-0f0992e82b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:30.489391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2d2b-9c86-4786-a7df-0f0992e82b59 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:30.489669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2d2b-9c86-4786-a7df-0f0992e82b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:30.489942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2d2b-9c86-4786-a7df-0f0992e82b59 None admin] PATH_INFO: `/v3/projects/a4369623e88e47428b14f0f10398a29f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:30.491103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d2d2b-9c86-4786-a7df-0f0992e82b59 None admin] RBAC: Authorizing `identity:get_project(project_id=a4369623e88e47428b14f0f10398a29f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:30.492502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d2d2b-9c86-4786-a7df-0f0992e82b59 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:30.494137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 610/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Aug 6 14:19:30 2022] GET /identity/v3/projects/a4369623e88e47428b14f0f10398a29f => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:19:30.500318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:19:30.556296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:30.595361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5bfd-dddb-42ec-baa0-1c7fd3bfa266 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:19:30.628597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0883762-3d74-4ae4-a0ed-0f4fc1f54c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:19:30.629115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883762-3d74-4ae4-a0ed-0f4fc1f54c21 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:19:30.629396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883762-3d74-4ae4-a0ed-0f4fc1f54c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:19:30.629685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883762-3d74-4ae4-a0ed-0f4fc1f54c21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:19:30.630460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0883762-3d74-4ae4-a0ed-0f4fc1f54c21 None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=a4369623e88e47428b14f0f10398a29f, user.id=a11feb327e134cfa81764c35dd905a37) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:19:30.630768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0883762-3d74-4ae4-a0ed-0f4fc1f54c21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:19:30.632227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0883762-3d74-4ae4-a0ed-0f4fc1f54c21 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:19:30.637834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 590/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:19:30 2022] GET /identity/v3/role_assignments?user.id=a11feb327e134cfa81764c35dd905a37&scope.project.id=a4369623e88e47428b14f0f10398a29f&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:19:40.483921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=76 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:19:40.635272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=38 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:25.428012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564e8f0-e281-4184-b760-d08df59cc519 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:25.428012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564e8f0-e281-4184-b760-d08df59cc519 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:25.428012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564e8f0-e281-4184-b760-d08df59cc519 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:25.429019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 611/1201] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:25.434967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1f6c-7fb4-4804-b414-c3125b17448f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:25.435471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1f6c-7fb4-4804-b414-c3125b17448f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:25.435567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1f6c-7fb4-4804-b414-c3125b17448f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:25.463294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e3dc1f6c-7fb4-4804-b414-c3125b17448f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:25.464983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e3dc1f6c-7fb4-4804-b414-c3125b17448f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:25.570823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 591/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:25.604509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564e8f0-e281-4184-b760-d08df59cc519 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:25.787229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564e8f0-e281-4184-b760-d08df59cc519 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:25.837753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:25.838503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:25.838899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:25.839185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:25.858138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:25.861322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:25.887411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:25.888666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:25.890525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:25.890525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:25.890525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:25.890525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:25.890525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 612/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:20:25.900980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc1f6c-7fb4-4804-b414-c3125b17448f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:26.038080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc1f6c-7fb4-4804-b414-c3125b17448f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:26.089584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:26.099226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23279093-dc60-4b5f-9043-d28767e1933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:26.099906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23279093-dc60-4b5f-9043-d28767e1933d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:26.100146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23279093-dc60-4b5f-9043-d28767e1933d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:26.100455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23279093-dc60-4b5f-9043-d28767e1933d None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:26.101131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23279093-dc60-4b5f-9043-d28767e1933d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:26.101444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23279093-dc60-4b5f-9043-d28767e1933d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:26.102857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23279093-dc60-4b5f-9043-d28767e1933d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:26.118913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 592/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:26.124855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:26.211280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:26.267479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693b586-0819-42f3-9212-41f0050dfc9b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:26.318603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834e226-8996-468c-825a-4b1fd758b576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:26.319253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834e226-8996-468c-825a-4b1fd758b576 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:26.319539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834e226-8996-468c-825a-4b1fd758b576 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:26.319818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834e226-8996-468c-825a-4b1fd758b576 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:26.320546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834e226-8996-468c-825a-4b1fd758b576 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:26.322039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834e226-8996-468c-825a-4b1fd758b576 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:26.352783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 613/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Aug 6 14:20:26 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:20:29.208558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccddd9-e1c9-403b-ba1f-ab8d513c7c06 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:29.209107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccddd9-e1c9-403b-ba1f-ab8d513c7c06 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:29.209107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccddd9-e1c9-403b-ba1f-ab8d513c7c06 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:29.212196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 593/1206] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:29.216110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608033f7-2d3e-4b26-a70b-4eb281578cb9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:29.216600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608033f7-2d3e-4b26-a70b-4eb281578cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:29.216736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608033f7-2d3e-4b26-a70b-4eb281578cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:29.240281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-608033f7-2d3e-4b26-a70b-4eb281578cb9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:29.241599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-608033f7-2d3e-4b26-a70b-4eb281578cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:29.330453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 614/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:29.361778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccddd9-e1c9-403b-ba1f-ab8d513c7c06 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:29.492509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccddd9-e1c9-403b-ba1f-ab8d513c7c06 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:29.532378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:29.533556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:29.533964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:29.534326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:29.547559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:20:29.553665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:29.555007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:29.555888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 594/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:20:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:29.561756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608033f7-2d3e-4b26-a70b-4eb281578cb9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:29.616884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:29.698523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608033f7-2d3e-4b26-a70b-4eb281578cb9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:29.743926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:29.744477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:29.744772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:29.745032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:29.745749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:29.746054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:29.747451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:29.756006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 615/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:20:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:20:29.761775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:29.824489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:29.882732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441edf55-112e-40f2-931d-619fad6b00ab None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:29.924630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59088045-f886-4688-8af4-663f114162dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:29.925791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088045-f886-4688-8af4-663f114162dd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:29.926230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088045-f886-4688-8af4-663f114162dd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:29.926514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088045-f886-4688-8af4-663f114162dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:29.937228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088045-f886-4688-8af4-663f114162dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:29.938688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088045-f886-4688-8af4-663f114162dd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-59088045-f886-4688-8af4-663f114162dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:29.956671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:29.957602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:29.959675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:29.959675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:29.959675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:29.959675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:29.959675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 595/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:29.964172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:30.075161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:30.077961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcf043-64f3-40a1-b40c-96b80edb32c6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:30.115927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:30.116552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:30.116835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:30.117112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:30.117822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:30.118109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:30.119510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:30.132209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 616/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:30.138152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59088045-f886-4688-8af4-663f114162dd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:30.253804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59088045-f886-4688-8af4-663f114162dd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:30.255406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:30.294225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:30.295000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:30.295313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:30.295655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:30.304949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:30.306571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:20:30.321677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:30.322808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:30.328308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 596/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:30.329076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:30.448135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0c8c3-a15f-423b-aafe-2166040209b6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:30.495507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:30.496381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:30.496707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:30.497018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:30.497018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:30.498211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:30.500722 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:30.541245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 617/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Aug 6 14:20:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:30.553470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:30.684148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f907-70c1-408f-8475-3df6a10cb811 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:30.745327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:30.746375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:30.746688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:30.746953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:30.782734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:30.839711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:30.842042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:30.965299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 597/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:20:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 14:20:30.971548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:30.992356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:31.079105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac0204-6821-4b3c-903d-5c22013d687f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:31.120339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:31.121244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:31.121576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:31.121894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:31.132895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:31.134532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:31.150424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:31.151560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:31.152867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:31.152867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:31.152867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:31.152867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:31.152867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 618/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:31.157389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:31.229559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:31.270664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c60fb-0829-4f0d-ae26-6ced2f19dbc1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:31.309312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:31.310223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:31.310741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:31.311138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:31.312043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:31.312555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:31.314215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:31.332347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 598/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:31.333574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:31.410600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:31.444857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174ba8-1764-4d65-bd4b-66a57f50b9e2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:31.482475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:31.483078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:31.483363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:31.483652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:31.491292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:31.492744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:20:31.508041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:31.509290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:31.510431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:31.516572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 619/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:20:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:31.518352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:31.628132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e664a63-a0d8-4768-9fd6-67601957210a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:31.655211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:31.672594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179f5d0-47f5-4065-81f3-0f046021bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:31.673630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179f5d0-47f5-4065-81f3-0f046021bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:31.674088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179f5d0-47f5-4065-81f3-0f046021bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:31.674608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179f5d0-47f5-4065-81f3-0f046021bfc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:31.675925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179f5d0-47f5-4065-81f3-0f046021bfc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:31.676482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179f5d0-47f5-4065-81f3-0f046021bfc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:31.678188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179f5d0-47f5-4065-81f3-0f046021bfc1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:31.694562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 599/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:31.704047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:31.833754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428211af-d1aa-4123-adc1-4d4ba1797449 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:31.868273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:31.874796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5887e5-50c3-4f40-b985-280d080a989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:31.875440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5887e5-50c3-4f40-b985-280d080a989e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:31.875716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5887e5-50c3-4f40-b985-280d080a989e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:31.875991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5887e5-50c3-4f40-b985-280d080a989e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:31.877598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5887e5-50c3-4f40-b985-280d080a989e None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=d607b8bbb8f047f0bbaf8dad33d47247) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:31.877933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5887e5-50c3-4f40-b985-280d080a989e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:31.879756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5887e5-50c3-4f40-b985-280d080a989e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:31.886696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 620/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:20:31 2022] GET /identity/v3/role_assignments?user.id=d607b8bbb8f047f0bbaf8dad33d47247&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:33.969767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8d1cf-3412-47d3-b986-e97615f87d84 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:33.971109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8d1cf-3412-47d3-b986-e97615f87d84 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:33.971109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8d1cf-3412-47d3-b986-e97615f87d84 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:33.972394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 600/1220] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:33.979229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00817-2a4a-4d1b-a8c1-54492c7bd415 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:33.979747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00817-2a4a-4d1b-a8c1-54492c7bd415 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:33.979894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00817-2a4a-4d1b-a8c1-54492c7bd415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:33.998327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-31d00817-2a4a-4d1b-a8c1-54492c7bd415 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:33.999541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-31d00817-2a4a-4d1b-a8c1-54492c7bd415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:34.077195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 621/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 14:20:34.100604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8d1cf-3412-47d3-b986-e97615f87d84 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:34.212241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8d1cf-3412-47d3-b986-e97615f87d84 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:34.246887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:34.247524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:34.247799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:34.248067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:34.258469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:20:34.264647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:34.265812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:34.266806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 601/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:20:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:34.273316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00817-2a4a-4d1b-a8c1-54492c7bd415 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:34.380560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00817-2a4a-4d1b-a8c1-54492c7bd415 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:34.409219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:34.420417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:34.421040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:34.421320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:34.421592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:34.422296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:34.422600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:34.424021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:34.432282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 622/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:20:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:34.439379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:34.489535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:34.557118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20ba3-fb6f-4cf5-9fc1-12e11948b626 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:34.596089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:34.596733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:34.597065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:34.597286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:34.607872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:34.609310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:34.624618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:34.625709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:34.626713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:34.626713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:34.626713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:34.626713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:34.626713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 602/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:34.634435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:34.745454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe0533-fce0-4ccd-badc-33e99c1b89f8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:34.780084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:34.780705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:34.780986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:34.781259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:34.781946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:34.782259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:34.783642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:34.795986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 623/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:34.802317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:34.918163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:34.925071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1bc32-bed0-4c50-9dfb-002b03100064 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:34.965216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:34.965838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:34.966115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:34.966387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:34.973916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:34.975476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:20:34.987194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:34.993704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:34.995157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 603/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:34.995792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:35.047971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:35.110166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e708d3b-ec71-48c5-947f-705320689517 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:35.148319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:35.148978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:35.149331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:35.149612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:35.150498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:35.150891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:35.152565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:35.172568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 624/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Aug 6 14:20:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:35.179634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:35.292086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbed31-0c5c-40e1-a4bb-bba26138483b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:35.329620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:35.330301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:35.330620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:35.330892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:35.342635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:35.344253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:35.358781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:35.359877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:35.361662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:35.361662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:35.361662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:35.361662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:35.361662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 604/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:35.366951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:35.484648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423e12e-ca60-4359-83b3-30d701c25d37 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:35.487810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:35.529296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:35.529941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:35.530204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:35.530482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:35.531196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:35.531527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:35.533381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:35.544825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 625/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:35.551974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:35.562195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:35.661937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab203-5efa-4919-957f-dae433486b24 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:35.697993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:35.698631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:35.698940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:35.699214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:35.706287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:35.707805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:20:35.721112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:35.722049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:35.723176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 605/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:20:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:20:35.728915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:35.840973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25290e6e-e451-44ab-bdbe-e9676cf6d54a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:35.882034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b311f-c29a-43f8-8c3f-4b6639e4b0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:35.882663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b311f-c29a-43f8-8c3f-4b6639e4b0aa None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:35.882975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b311f-c29a-43f8-8c3f-4b6639e4b0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:35.883321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b311f-c29a-43f8-8c3f-4b6639e4b0aa None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:35.883978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b311f-c29a-43f8-8c3f-4b6639e4b0aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:35.884280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b311f-c29a-43f8-8c3f-4b6639e4b0aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:35.885709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b311f-c29a-43f8-8c3f-4b6639e4b0aa None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:35.898288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 626/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:20:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:35.904376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:35.974647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:36.015711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7130d0-149b-4208-9185-5b4b1eb89f01 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:36.051429 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5f021-8aa2-4b9a-a744-e27c6ebbd46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:36.052278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5f021-8aa2-4b9a-a744-e27c6ebbd46c None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:36.052671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5f021-8aa2-4b9a-a744-e27c6ebbd46c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:36.052976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5f021-8aa2-4b9a-a744-e27c6ebbd46c None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/d607b8bbb8f047f0bbaf8dad33d47247/roles/c5d40ea25aa5440d8170744359aaa18a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:36.069158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5f021-8aa2-4b9a-a744-e27c6ebbd46c None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=d607b8bbb8f047f0bbaf8dad33d47247, role_id=c5d40ea25aa5440d8170744359aaa18a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:36.070786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5f021-8aa2-4b9a-a744-e27c6ebbd46c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:36.084120 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 606/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:20:35 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/d607b8bbb8f047f0bbaf8dad33d47247/roles/c5d40ea25aa5440d8170744359aaa18a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:20:38.064615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8751b9-4383-4cce-9c20-9a7387a88e16 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:38.065189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8751b9-4383-4cce-9c20-9a7387a88e16 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:38.065189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8751b9-4383-4cce-9c20-9a7387a88e16 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:38.066497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 627/1233] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:38.071597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509fccf-7b9d-433c-b150-d3825cb8cfbb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:38.071982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509fccf-7b9d-433c-b150-d3825cb8cfbb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:38.072150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509fccf-7b9d-433c-b150-d3825cb8cfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:38.089876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6509fccf-7b9d-433c-b150-d3825cb8cfbb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:38.091091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6509fccf-7b9d-433c-b150-d3825cb8cfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:38.160041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 607/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:38.184078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8751b9-4383-4cce-9c20-9a7387a88e16 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:38.212558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:38.294827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8751b9-4383-4cce-9c20-9a7387a88e16 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:38.337640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:38.338312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:38.338589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:38.338858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:38.350851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:20:38.356419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:38.357524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:38.358376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 628/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Aug 6 14:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:38.364231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509fccf-7b9d-433c-b150-d3825cb8cfbb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:38.419110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:38.477738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509fccf-7b9d-433c-b150-d3825cb8cfbb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:38.520385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:38.521064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:38.521524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:38.521884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:38.523111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:38.523499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:38.525251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:38.540940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 608/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Aug 6 14:20:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:20:38.542144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:38.629937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:38.648747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777dd2e-e626-445a-b319-38f92d63d06c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:38.689281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:38.689850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:38.690131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:38.690404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:38.701493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:38.703257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:38.720865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:38.721778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:38.722665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:38.722665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:38.722665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:38.722665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:38.722665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 629/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:20:38.727836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:38.843523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3acac-f886-4669-a0ff-e5e992e00b99 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:38.854427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:38.882766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:38.883538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:38.883827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:38.884104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:38.884798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:38.885124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:38.886520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:38.899198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 609/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:38.905165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:39.032394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e671e1-f5b3-442a-bbcf-976703b2f0df None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:39.068126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:39.068940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:39.069111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:39.069282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:39.072413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:39.078528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:39.080693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:20:39.091946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:39.093075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:39.094008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 630/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:39.099427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:39.203371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c80625-df68-4549-854f-530a7a7ebebb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:39.246269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:39.247238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:39.247804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:39.248080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:39.249318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:39.249821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:39.251948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:39.272814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 610/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Aug 6 14:20:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:39.278981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:39.395282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ce30-d716-4cb9-a02d-4344cb8223b7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:39.434630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:39.435294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:39.435621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:39.435843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:39.506480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:39.514241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:39.515752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:39.631564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 631/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:20:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 14:20:39.637444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:39.667673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:39.747960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58600a5-3b88-473c-b3f5-e5b030ea6c71 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:39.786051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:39.786815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:39.787149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:39.787472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:39.799972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:39.801901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:39.819012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:39.819968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:39.821113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:39.821113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:39.821113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:39.821113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:39.821113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 611/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:39.826773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:39.937934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:39.944592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb99fb7-1985-4b9f-bbcb-75ae4ed96e34 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:39.980196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12121d-5007-474e-a915-735680879977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:39.980959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12121d-5007-474e-a915-735680879977 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:39.981245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12121d-5007-474e-a915-735680879977 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:39.981544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12121d-5007-474e-a915-735680879977 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:39.982317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12121d-5007-474e-a915-735680879977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:39.982628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12121d-5007-474e-a915-735680879977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:39.984129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12121d-5007-474e-a915-735680879977 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:39.998198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 632/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:40.003909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:40.092643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:40.116878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e935411-16b5-406d-90a7-42932410ff93 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:40.151206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:40.151849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:40.152129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:40.152404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:40.159494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:40.160936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:20:40.172655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:40.174326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:40.175591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 612/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:20:40.180694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12121d-5007-474e-a915-735680879977 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:40.292501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12121d-5007-474e-a915-735680879977 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:40.324857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec25766-1ad3-4f2d-82d3-a54df5b0fa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:40.325468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec25766-1ad3-4f2d-82d3-a54df5b0fa78 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:40.325744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec25766-1ad3-4f2d-82d3-a54df5b0fa78 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:40.326015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec25766-1ad3-4f2d-82d3-a54df5b0fa78 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:40.326740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec25766-1ad3-4f2d-82d3-a54df5b0fa78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:40.327069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec25766-1ad3-4f2d-82d3-a54df5b0fa78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:40.328462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec25766-1ad3-4f2d-82d3-a54df5b0fa78 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:40.340593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 633/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:20:40.347130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:40.450888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3990ce9-b85d-47a9-bfd6-676df0b0ab26 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:40.487042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42df6c-a8c5-4b3c-9a39-09e6369b13e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:40.487651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42df6c-a8c5-4b3c-9a39-09e6369b13e4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:40.487929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42df6c-a8c5-4b3c-9a39-09e6369b13e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:40.488197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42df6c-a8c5-4b3c-9a39-09e6369b13e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:40.488972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42df6c-a8c5-4b3c-9a39-09e6369b13e4 None admin] RBAC: Adding query filter params (role.id=c5d40ea25aa5440d8170744359aaa18a, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=d607b8bbb8f047f0bbaf8dad33d47247) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:40.489276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42df6c-a8c5-4b3c-9a39-09e6369b13e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:40.490761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42df6c-a8c5-4b3c-9a39-09e6369b13e4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:40.496983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 613/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:20:40 2022] GET /identity/v3/role_assignments?user.id=d607b8bbb8f047f0bbaf8dad33d47247&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=c5d40ea25aa5440d8170744359aaa18a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:42.710228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df5d4-304b-4848-82d1-e6aa2e915d64 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:42.710768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df5d4-304b-4848-82d1-e6aa2e915d64 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:42.710921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df5d4-304b-4848-82d1-e6aa2e915d64 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:42.712516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 634/1247] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:42.718052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63001e75-a99e-456d-baa8-4da6161b0cc9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:42.718578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63001e75-a99e-456d-baa8-4da6161b0cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:42.718578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63001e75-a99e-456d-baa8-4da6161b0cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:42.725137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:42.738428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-63001e75-a99e-456d-baa8-4da6161b0cc9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:42.739762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-63001e75-a99e-456d-baa8-4da6161b0cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:42.822841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 614/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:42.847613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df5d4-304b-4848-82d1-e6aa2e915d64 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:42.868490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:42.959999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df5d4-304b-4848-82d1-e6aa2e915d64 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:42.997421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:42.998008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:42.998286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:42.998554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:43.014891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:43.015907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:20:43.016941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:43.018275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 635/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:20:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:43.024370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63001e75-a99e-456d-baa8-4da6161b0cc9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:43.133779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63001e75-a99e-456d-baa8-4da6161b0cc9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:43.165444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:43.171293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:43.171913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:43.172193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:43.172467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:43.173793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:43.174095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:43.175769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:43.185476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 615/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:20:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:43.191436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:43.275057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:43.299723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157af7ea-6a3b-4d20-ae1c-07e95f23dda8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:43.336116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:43.336767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:43.337054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:43.337361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:43.349498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:43.350944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:43.365646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:43.366725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:43.367687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:43.367687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:43.367687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:43.367687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:43.367687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 636/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:20:43.372685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:43.483984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6047b14-1967-4829-9f21-f043e44d0de5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:43.529502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:43.530313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:43.530723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:43.531223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:43.532857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:43.533459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:43.535352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:43.553010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 616/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:20:43.559664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:43.676468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516748f-484d-4067-b1af-6cffc51f4ab8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:43.704686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:43.712761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:43.713419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:43.713718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:43.713973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:43.721989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:43.723523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:20:43.736666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:43.737687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:43.738626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 637/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:43.744593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:43.802948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:43.858878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7032e7-f16f-4804-bfc3-2378b0ec7021 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:43.898374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:43.899024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:43.899305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:43.899573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:43.900423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:43.900657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:43.902518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:43.920715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 617/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Aug 6 14:20:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:43.927325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:44.044238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6baa61-c258-4109-8015-1672be021ea0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:44.083678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:44.084324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:44.084636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:44.084973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:44.150411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:44.164706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:44.166150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:44.288825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 638/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:20:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 14:20:44.299344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:44.435426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c4054-7287-4244-bd63-163afa36c978 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:44.448894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:44.471164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:44.471811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:44.472087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:44.472356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:44.482490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:44.484002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:44.499733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:44.505844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:44.506982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:44.506982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:44.506982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:44.506982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:44.506982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 618/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:44.507322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:44.605113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:44.621262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5f84-4ae7-4b75-845f-d632f46e23a1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:44.657973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:44.658603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:44.658902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:44.659187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:44.659879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:44.660181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:44.661575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:44.674156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 639/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:44.687005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:44.791502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd7659-bca1-4add-9fac-222df3eb59c9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:44.826744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:44.827541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:44.827828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:44.828126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:44.835599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:44.837095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:20:44.849153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:44.850247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:44.851216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:44.851659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 619/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:44.860537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:44.972845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be068bb-c444-4a19-a436-591965bc58d2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:45.009866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d96a4-c557-40b2-b576-c0c6492de51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:45.010624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d96a4-c557-40b2-b576-c0c6492de51d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:45.010926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d96a4-c557-40b2-b576-c0c6492de51d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:45.011208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d96a4-c557-40b2-b576-c0c6492de51d None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:45.012140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d96a4-c557-40b2-b576-c0c6492de51d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:45.012501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d96a4-c557-40b2-b576-c0c6492de51d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:45.014020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d96a4-c557-40b2-b576-c0c6492de51d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:45.029453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 640/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:20:45.037015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:45.063187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:45.146300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b2a16-20f7-4df0-a761-d4de2a8ed157 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:45.184372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25b94-c467-490f-aab8-7ceb98f9ce2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:45.185232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25b94-c467-490f-aab8-7ceb98f9ce2a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:45.185537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25b94-c467-490f-aab8-7ceb98f9ce2a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:45.185917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25b94-c467-490f-aab8-7ceb98f9ce2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:45.186844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad25b94-c467-490f-aab8-7ceb98f9ce2a None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=d607b8bbb8f047f0bbaf8dad33d47247) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:45.187332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad25b94-c467-490f-aab8-7ceb98f9ce2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:45.189756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad25b94-c467-490f-aab8-7ceb98f9ce2a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:45.210903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 620/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:20:45 2022] GET /identity/v3/role_assignments?user.id=d607b8bbb8f047f0bbaf8dad33d47247&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:47.194606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4b049-c89d-48b5-81cd-8fd9f27064bb None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:47.195159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4b049-c89d-48b5-81cd-8fd9f27064bb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:47.195159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4b049-c89d-48b5-81cd-8fd9f27064bb None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:47.196505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 641/1261] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:47.201620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f9c5-b1f5-41dd-99eb-40820bbeb8d0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:47.202056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f9c5-b1f5-41dd-99eb-40820bbeb8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:47.202157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f9c5-b1f5-41dd-99eb-40820bbeb8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:47.220965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c771f9c5-b1f5-41dd-99eb-40820bbeb8d0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:47.222141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c771f9c5-b1f5-41dd-99eb-40820bbeb8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:47.285568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 621/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:47.307213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4b049-c89d-48b5-81cd-8fd9f27064bb None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:47.317751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:47.405748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4b049-c89d-48b5-81cd-8fd9f27064bb None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:47.438924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:47.439551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:47.439830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:47.440103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:47.449870 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:20:47.455314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:47.456372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:47.457366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 642/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:20:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:47.462263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771f9c5-b1f5-41dd-99eb-40820bbeb8d0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:47.479831 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:47.574105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771f9c5-b1f5-41dd-99eb-40820bbeb8d0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:47.606508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:47.607214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:47.607494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:47.607763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:47.608463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:47.608761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:47.610127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:47.617916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 622/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:20:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:47.623667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:47.688236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:47.720642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2143c38-861f-4351-b9b1-73c0f65b1baa None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:47.752682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:47.753306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:47.753578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:47.753846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:47.763405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:47.764811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:47.779515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:47.780559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:47.781664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:47.781664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:47.781664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:47.781664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:47.781664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 643/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:47.786306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:47.865485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:47.884527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274c760-5e72-4e95-bfec-5c983049d96e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:47.915946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:47.916553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:47.916829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:47.917099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:47.917776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:47.918103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:47.919487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:47.932329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 623/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:47.937903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:48.036098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5952c-1ad0-443e-a1db-e768c5bc81ff None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:48.058530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:48.073953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:48.074586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:48.074862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:48.075167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:48.082366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:48.083816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:20:48.094560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:48.095682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:48.096662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 644/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:48.101500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:48.197444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5124433-b24a-405e-bf2f-f398d7aad431 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:48.229381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:48.229985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:48.230260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:48.230531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:48.231328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:48.231631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:48.233029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:48.241786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:48.250149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 624/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Aug 6 14:20:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:48.255697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:48.354647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7194-950e-495f-97bb-648e0104bda7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:48.386498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:48.387146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:48.387425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:48.387697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:48.397699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:48.399142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:48.413166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:48.414188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:48.415343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:48.415343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:48.415343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:48.415343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:48.415343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 645/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 14:20:48.419934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:48.522057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ffc2e-998d-4dcf-9b89-995389422ec5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:48.556712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:48.557323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:48.557600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:48.557872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:48.558552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:48.558862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:48.560245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:48.571012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 625/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:20:48.576612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:48.607893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:48.672984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6eda0-fc0a-462c-bc5a-1690d255fe40 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:48.704987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:48.705600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:48.705875 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:48.706145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:48.712478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:48.713882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:20:48.725253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:48.726327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:48.727443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 646/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:20:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:48.731995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:48.787930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:48.827850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850ed6e-0bba-4443-9ad6-cc8173f1ae16 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:48.861269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d98ec-124e-4da7-8aa8-2c5b003096b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:48.861858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d98ec-124e-4da7-8aa8-2c5b003096b3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:48.862138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d98ec-124e-4da7-8aa8-2c5b003096b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:48.862412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d98ec-124e-4da7-8aa8-2c5b003096b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:48.863181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d98ec-124e-4da7-8aa8-2c5b003096b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:48.863486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d98ec-124e-4da7-8aa8-2c5b003096b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:48.864913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d98ec-124e-4da7-8aa8-2c5b003096b3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:48.876436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 626/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:20:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:48.882242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:48.971082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:48.980095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd502-98c0-40ca-9162-297710b51027 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:49.011966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6484076-e276-4d09-b06b-d5293b69ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:49.012608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6484076-e276-4d09-b06b-d5293b69ea6c None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:49.012882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6484076-e276-4d09-b06b-d5293b69ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:49.013151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6484076-e276-4d09-b06b-d5293b69ea6c None admin] PATH_INFO: `/v3/projects/515e01743cac40eca8e54c926c4dff29/users/d607b8bbb8f047f0bbaf8dad33d47247/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:49.015189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6484076-e276-4d09-b06b-d5293b69ea6c None admin] RBAC: Authorizing `identity:create_grant(project_id=515e01743cac40eca8e54c926c4dff29, user_id=d607b8bbb8f047f0bbaf8dad33d47247, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:49.017340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6484076-e276-4d09-b06b-d5293b69ea6c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:49.029257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 647/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Aug 6 14:20:48 2022] PUT /identity/v3/projects/515e01743cac40eca8e54c926c4dff29/users/d607b8bbb8f047f0bbaf8dad33d47247/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:20:50.990820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788d460-3bae-465c-be58-465aa9fa3275 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:50.991371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788d460-3bae-465c-be58-465aa9fa3275 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:50.991520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788d460-3bae-465c-be58-465aa9fa3275 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:50.992878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 627/1274] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:50.997961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf73be-11b2-437c-a25c-6868d3d0bd7d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:50.998367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf73be-11b2-437c-a25c-6868d3d0bd7d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:50.998514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf73be-11b2-437c-a25c-6868d3d0bd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:51.016726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2bdf73be-11b2-437c-a25c-6868d3d0bd7d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:51.017905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2bdf73be-11b2-437c-a25c-6868d3d0bd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:51.081931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 648/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:51.104289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788d460-3bae-465c-be58-465aa9fa3275 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:51.204589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788d460-3bae-465c-be58-465aa9fa3275 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:51.222087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:51.239282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:51.239922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:51.240200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:51.240473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:51.250133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 14:20:51.255088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:51.256203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:51.257272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 628/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Aug 6 14:20:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:51.262348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf73be-11b2-437c-a25c-6868d3d0bd7d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:51.341119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:51.359704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf73be-11b2-437c-a25c-6868d3d0bd7d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:51.391965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:51.392568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:51.392844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:51.393112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:51.393803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:51.394099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:51.395468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:51.403232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 649/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Aug 6 14:20:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:51.409396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:51.507979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec795d4-8653-4b17-a47c-fca4276f1adb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:51.542093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:51.542726 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:51.543033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:51.543308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:51.553654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:51.555139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:51.568754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:51.569789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:51.570916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:51.570916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:51.570916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:51.570916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:51.570916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 629/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:51.575643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:51.677152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b7f5c-ec49-4d70-9b32-dd44ebf1d1a6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:51.709495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:51.710106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:51.710389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:51.710670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:51.711382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:51.711687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:51.713031 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:51.724966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 650/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:51.730619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:51.759691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:51.826346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9419063-8613-48ac-95ff-1efc5f6fdbcf None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:51.859039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:51.859674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:51.859950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:51.860249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:51.867401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:51.868830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 14:20:51.879591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:51.880646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:51.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 630/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:51.886435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:51.890189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:51.985602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1195b-37e6-4345-a20a-56eda18cba50 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:52.018395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:52.019018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:52.019299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:52.019566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:52.020333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:52.020634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:52.022022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:52.038166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 651/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Aug 6 14:20:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:52.044065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:52.130680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:52.140351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33b728-04ea-49b2-ad5e-14030c862e49 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:52.172304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:52.172934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:52.173211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:52.173484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:52.237880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:52.239275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:52.323169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:52.333515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 631/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Aug 6 14:20:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 14:20:52.339293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:52.398155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:52.436380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23348e78-5917-4f8a-928c-21c41e586f63 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:52.469125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:52.469762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:52.470041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:52.470316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:52.480033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:52.481457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 14:20:52.498045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:52.499098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:52.500012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 14:20:52.500012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 14:20:52.500012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 14:20:52.500012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:52.500012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 652/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:52.506031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:52.601298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a2e1-a2e9-4857-8dc3-813353a63d05 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:52.633979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:52.634582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:52.634861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:52.635160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:52.635830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:52.636132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:52.637482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:52.648257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 632/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Aug 6 14:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:52.653744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:52.751516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4112e-01c5-4000-ab76-0f89fdac62c3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:52.764349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:52.784618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:52.785236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:52.785515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:52.785788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:52.792226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:52.793641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 14:20:52.805547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:52.806568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Aug 06 14:20:52.807546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 653/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Aug 6 14:20:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:52.812290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:52.861452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:52.909008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21573593-246c-4a43-9022-ef7a71b5243f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:52.940867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b541883-93d1-4e1b-9eda-455d799635d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:52.941470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b541883-93d1-4e1b-9eda-455d799635d8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:52.941747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b541883-93d1-4e1b-9eda-455d799635d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:52.942017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b541883-93d1-4e1b-9eda-455d799635d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:52.942772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b541883-93d1-4e1b-9eda-455d799635d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:52.943105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b541883-93d1-4e1b-9eda-455d799635d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:52.944511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b541883-93d1-4e1b-9eda-455d799635d8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:52.955813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 633/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Aug 6 14:20:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:52.962054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:53.057841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf229d7-7fc1-43b6-aa77-8ee3babe52b1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:53.089629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ba739-f787-4a7c-a591-d618e65b314c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:53.090223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba739-f787-4a7c-a591-d618e65b314c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:53.090503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba739-f787-4a7c-a591-d618e65b314c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:53.090783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba739-f787-4a7c-a591-d618e65b314c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:53.091578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ba739-f787-4a7c-a591-d618e65b314c None admin] RBAC: Adding query filter params (role.id=93fd86974d4f46edb7f9d9ff528f009d, scope.project.id=515e01743cac40eca8e54c926c4dff29, user.id=d607b8bbb8f047f0bbaf8dad33d47247) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:53.091885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ba739-f787-4a7c-a591-d618e65b314c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:53.093289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ba739-f787-4a7c-a591-d618e65b314c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:53.099432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 654/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Aug 6 14:20:52 2022] GET /identity/v3/role_assignments?user.id=d607b8bbb8f047f0bbaf8dad33d47247&scope.project.id=515e01743cac40eca8e54c926c4dff29&role.id=93fd86974d4f46edb7f9d9ff528f009d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:55.111545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15909-a5eb-45b8-9569-b877a04aae51 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:55.112063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15909-a5eb-45b8-9569-b877a04aae51 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:55.112063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15909-a5eb-45b8-9569-b877a04aae51 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:55.113344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 634/1288] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:55.118291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb35e-aa7b-4295-a7fa-93555766a5ec None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:55.118703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb35e-aa7b-4295-a7fa-93555766a5ec None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:55.118894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb35e-aa7b-4295-a7fa-93555766a5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:55.136923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c0deb35e-aa7b-4295-a7fa-93555766a5ec None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:55.138124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c0deb35e-aa7b-4295-a7fa-93555766a5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:55.166856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:55.203411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 655/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:55.225277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15909-a5eb-45b8-9569-b877a04aae51 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:55.323301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15909-a5eb-45b8-9569-b877a04aae51 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:55.333381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:55.356380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:55.357005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:55.357279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:55.357548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:55.358159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:55.359541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:20:55.368593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:20:55.369766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:20:55.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 635/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:20:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:20:55.376466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0deb35e-aa7b-4295-a7fa-93555766a5ec None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:55.470887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0deb35e-aa7b-4295-a7fa-93555766a5ec None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:55.507376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760fc7f-e83e-43c6-ad76-b84c61c7e3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:55.508046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760fc7f-e83e-43c6-ad76-b84c61c7e3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:55.508364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760fc7f-e83e-43c6-ad76-b84c61c7e3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:55.508664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760fc7f-e83e-43c6-ad76-b84c61c7e3f6 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:55.509407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760fc7f-e83e-43c6-ad76-b84c61c7e3f6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:55.509814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760fc7f-e83e-43c6-ad76-b84c61c7e3f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:55.511407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760fc7f-e83e-43c6-ad76-b84c61c7e3f6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:55.517349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 656/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Aug 6 14:20:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:55.524072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:55.624945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833011-15fe-4285-a73e-b1d6bafc5342 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:55.657423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cbcae-a886-4bf1-9a31-dbe1f3dc8fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:55.658031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cbcae-a886-4bf1-9a31-dbe1f3dc8fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:55.658311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cbcae-a886-4bf1-9a31-dbe1f3dc8fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:55.658581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cbcae-a886-4bf1-9a31-dbe1f3dc8fa7 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:55.659273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cbcae-a886-4bf1-9a31-dbe1f3dc8fa7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:20:55.659575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cbcae-a886-4bf1-9a31-dbe1f3dc8fa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:55.660941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cbcae-a886-4bf1-9a31-dbe1f3dc8fa7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:55.665200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 636/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Aug 6 14:20:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:20:57.614076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64aa77-9065-41f3-98bf-1aa92750b49d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:57.614623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64aa77-9065-41f3-98bf-1aa92750b49d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:57.614623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64aa77-9065-41f3-98bf-1aa92750b49d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:57.615993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 657/1293] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:57.620969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc1da-2789-4a54-b02e-6eb3088fe693 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:57.621384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc1da-2789-4a54-b02e-6eb3088fe693 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:57.621521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc1da-2789-4a54-b02e-6eb3088fe693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:57.639625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f63dc1da-2789-4a54-b02e-6eb3088fe693 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:57.640791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f63dc1da-2789-4a54-b02e-6eb3088fe693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:57.668332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:57.704485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 637/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:20:57.726034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64aa77-9065-41f3-98bf-1aa92750b49d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:20:57.756832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:20:57.824808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64aa77-9065-41f3-98bf-1aa92750b49d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:20:57.857442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b51cb-106a-43d1-876d-ee42010a5175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:20:57.858058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b51cb-106a-43d1-876d-ee42010a5175 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:57.858337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b51cb-106a-43d1-876d-ee42010a5175 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:57.858609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b51cb-106a-43d1-876d-ee42010a5175 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:57.859322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b51cb-106a-43d1-876d-ee42010a5175 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:20:57.861160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b51cb-106a-43d1-876d-ee42010a5175 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:20:57.875293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 658/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Aug 6 14:20:57 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:20:59.900867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1bddd-e594-4327-a317-b7bc442e4db2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:59.901304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1bddd-e594-4327-a317-b7bc442e4db2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:59.901422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1bddd-e594-4327-a317-b7bc442e4db2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:59.902793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 638/1296] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:20:59.908292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e04e-e6a3-4579-a7b4-e7d4a56f4cdc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:20:59.908746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e04e-e6a3-4579-a7b4-e7d4a56f4cdc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:20:59.908851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e04e-e6a3-4579-a7b4-e7d4a56f4cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:20:59.926553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0e75e04e-e6a3-4579-a7b4-e7d4a56f4cdc None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:20:59.927753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0e75e04e-e6a3-4579-a7b4-e7d4a56f4cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:20:59.996988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 659/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:20:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:21:00.026283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1bddd-e594-4327-a317-b7bc442e4db2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:21:00.130755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1bddd-e594-4327-a317-b7bc442e4db2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:21:00.164272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:21:00.164900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:00.165177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:00.165447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:00.166058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:21:00.167455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:21:00.176214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:21:00.177359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:21:00.178502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 639/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:21:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:21:00.183383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e04e-e6a3-4579-a7b4-e7d4a56f4cdc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:21:00.201826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:21:00.281301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e04e-e6a3-4579-a7b4-e7d4a56f4cdc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:21:00.314122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:21:00.314725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:00.315037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:00.315316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:00.315986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] RBAC: Adding query filter params (name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:21:00.316293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:21:00.317644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:21:00.322620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 660/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Aug 6 14:21:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:21:00.328362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:21:00.359481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:21:00.425632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996259eb-b51b-487d-8e53-1750cf712f58 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:21:00.458046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b9282-8996-4ce4-907c-880d200a5fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:21:00.458651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9282-8996-4ce4-907c-880d200a5fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:00.458952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9282-8996-4ce4-907c-880d200a5fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:00.459227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9282-8996-4ce4-907c-880d200a5fe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:00.459959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9282-8996-4ce4-907c-880d200a5fe9 None admin] RBAC: Adding query filter params (interface=public, service_id=a52d6ae042de43e7a2e25fe49a454fc7, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:21:00.460264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9282-8996-4ce4-907c-880d200a5fe9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:21:00.461613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9282-8996-4ce4-907c-880d200a5fe9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:21:00.466427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 640/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Aug 6 14:21:00 2022] GET /identity/v3/endpoints?service_id=a52d6ae042de43e7a2e25fe49a454fc7&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:21:00.472118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:21:00.562277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:21:00.580659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61540e-b2c9-4769-b287-7c6ee4200dae None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:21:00.613123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420318fc-4c5c-4d48-a354-40387d68427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:21:00.613727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420318fc-4c5c-4d48-a354-40387d68427f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:00.614002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420318fc-4c5c-4d48-a354-40387d68427f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:00.614273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420318fc-4c5c-4d48-a354-40387d68427f None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:00.614925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420318fc-4c5c-4d48-a354-40387d68427f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:21:00.616259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420318fc-4c5c-4d48-a354-40387d68427f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:21:00.622784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 661/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Aug 6 14:21:00 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:21:02.581109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7dabf-2182-4271-aef8-c1b9e9532a30 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:02.581701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7dabf-2182-4271-aef8-c1b9e9532a30 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:02.581701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7dabf-2182-4271-aef8-c1b9e9532a30 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:02.583011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 641/1302] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:21:02.588031 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3d289-0dbc-46f4-ad73-79d28ffbc6e0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:02.588443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3d289-0dbc-46f4-ad73-79d28ffbc6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:02.588588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3d289-0dbc-46f4-ad73-79d28ffbc6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:02.606137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cbc3d289-0dbc-46f4-ad73-79d28ffbc6e0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:21:02.607360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cbc3d289-0dbc-46f4-ad73-79d28ffbc6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:21:02.672567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 662/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:21:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 14:21:02.695748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7dabf-2182-4271-aef8-c1b9e9532a30 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:21:02.787447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:21:02.794294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7dabf-2182-4271-aef8-c1b9e9532a30 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:21:02.827397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:21:02.828023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:02.828305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:02.828602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:02.829219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:21:02.830579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 14:21:02.839255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 14:21:02.840578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Aug 06 14:21:02.841869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 642/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Aug 6 14:21:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 14:21:02.846293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3d289-0dbc-46f4-ad73-79d28ffbc6e0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:21:02.920089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:21:02.943507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3d289-0dbc-46f4-ad73-79d28ffbc6e0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:21:02.975224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ba590-e61d-4d8f-955e-2ca21f1106e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:21:02.975829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ba590-e61d-4d8f-955e-2ca21f1106e8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:02.976109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ba590-e61d-4d8f-955e-2ca21f1106e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:02.976382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ba590-e61d-4d8f-955e-2ca21f1106e8 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:02.977040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ba590-e61d-4d8f-955e-2ca21f1106e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:21:02.977339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ba590-e61d-4d8f-955e-2ca21f1106e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:21:02.978680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ba590-e61d-4d8f-955e-2ca21f1106e8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:21:02.983638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 663/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Aug 6 14:21:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:21:02.989699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:21:03.084336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb0959-0b46-458f-8127-5b468a34fd19 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:21:03.116198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27bb24-9c66-4f2d-96a8-1e0088406308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:21:03.116736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27bb24-9c66-4f2d-96a8-1e0088406308 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:21:03.117011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27bb24-9c66-4f2d-96a8-1e0088406308 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:21:03.117278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27bb24-9c66-4f2d-96a8-1e0088406308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:21:03.117970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27bb24-9c66-4f2d-96a8-1e0088406308 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:21:03.119350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27bb24-9c66-4f2d-96a8-1e0088406308 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:21:03.139095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 643/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:21:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:21:12.980698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=30 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:21:13.134496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:05.659110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcd1c2-206b-4c13-8332-98d0fefb91ac None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:05.660048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcd1c2-206b-4c13-8332-98d0fefb91ac None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:05.660471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcd1c2-206b-4c13-8332-98d0fefb91ac None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:05.667476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e042-1e57-4d27-a968-d5db445edb90 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:05.668298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e042-1e57-4d27-a968-d5db445edb90 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:05.668725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e042-1e57-4d27-a968-d5db445edb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:05.672174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 664/1307] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:05.745232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2d79e042-1e57-4d27-a968-d5db445edb90 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:05.746703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2d79e042-1e57-4d27-a968-d5db445edb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:05.935894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 644/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:05.976217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5f13-f968-468a-ac38-9456b8021de8 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:05.976500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5f13-f968-468a-ac38-9456b8021de8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:05.976844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5f13-f968-468a-ac38-9456b8021de8 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:05.978102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 665/1309] 10.222.0.17 () {54 vars in 965 bytes} [Sat Aug 6 14:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:05.987110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae118b95-49f9-4058-a29b-183703d8c403 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:05.987552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae118b95-49f9-4058-a29b-183703d8c403 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:05.987733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae118b95-49f9-4058-a29b-183703d8c403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:06.112065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ae118b95-49f9-4058-a29b-183703d8c403 None None] MFA Rules not processed for user `cd235744a7ee462eb13e383930de4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:06.113699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ae118b95-49f9-4058-a29b-183703d8c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:06.136743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:06.195196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 645/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Sat Aug 6 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:06.207483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5f13-f968-468a-ac38-9456b8021de8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:06.209356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8c5e5f13-f968-468a-ac38-9456b8021de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:06.339655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5f13-f968-468a-ac38-9456b8021de8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:06.417372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe5bb9-b850-4ed7-963f-57b8380b8e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:06.417989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe5bb9-b850-4ed7-963f-57b8380b8e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:06.418264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe5bb9-b850-4ed7-963f-57b8380b8e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:06.418530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe5bb9-b850-4ed7-963f-57b8380b8e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:06.420158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-adfe5bb9-b850-4ed7-963f-57b8380b8e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:06.477032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:06.534229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe5bb9-b850-4ed7-963f-57b8380b8e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:06.535847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe5bb9-b850-4ed7-963f-57b8380b8e95 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:06.556648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 666/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:08.879650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac6f2-f7a2-40ac-8ccc-a596497e90f2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:08.880155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac6f2-f7a2-40ac-8ccc-a596497e90f2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:08.880258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac6f2-f7a2-40ac-8ccc-a596497e90f2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:08.881701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 646/1312] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:08.887041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd5e17-efe6-4c83-84e2-49f68734c4d2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:08.887497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd5e17-efe6-4c83-84e2-49f68734c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:08.887601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd5e17-efe6-4c83-84e2-49f68734c4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:08.973590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-68cd5e17-efe6-4c83-84e2-49f68734c4d2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:08.974791 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-68cd5e17-efe6-4c83-84e2-49f68734c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:09.087368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 667/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:09.127131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac6f2-f7a2-40ac-8ccc-a596497e90f2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:09.128939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4eeac6f2-f7a2-40ac-8ccc-a596497e90f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:09.246781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac6f2-f7a2-40ac-8ccc-a596497e90f2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:09.324427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:09.327291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72999ee0-6dc5-4190-a746-c0dfabb396ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:09.328183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999ee0-6dc5-4190-a746-c0dfabb396ce service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:09.328686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999ee0-6dc5-4190-a746-c0dfabb396ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:09.329137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999ee0-6dc5-4190-a746-c0dfabb396ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:09.330819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-72999ee0-6dc5-4190-a746-c0dfabb396ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:09.465643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72999ee0-6dc5-4190-a746-c0dfabb396ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:09.467458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72999ee0-6dc5-4190-a746-c0dfabb396ce service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:09.488010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 647/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:11.778433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b52a-0950-4027-bb65-e2d3238b55cd None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:11.779000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b52a-0950-4027-bb65-e2d3238b55cd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:11.779000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b52a-0950-4027-bb65-e2d3238b55cd None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:11.780887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 668/1315] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:11.786026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42451-e647-4729-bd17-23f140e9ab8e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:11.786427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42451-e647-4729-bd17-23f140e9ab8e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:11.786574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42451-e647-4729-bd17-23f140e9ab8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:11.850847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5fc42451-e647-4729-bd17-23f140e9ab8e None None] MFA Rules not processed for user `303ba9e42896448ca649a81fa5154966`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:11.852138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5fc42451-e647-4729-bd17-23f140e9ab8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:11.906393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:11.949532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 648/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:11.989534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b52a-0950-4027-bb65-e2d3238b55cd None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:11.991125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-af78b52a-0950-4027-bb65-e2d3238b55cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:12.000802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:12.093259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b52a-0950-4027-bb65-e2d3238b55cd None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:12.163495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd230c-959b-4cb7-8672-5418b6cb1b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:12.164101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd230c-959b-4cb7-8672-5418b6cb1b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:12.164379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd230c-959b-4cb7-8672-5418b6cb1b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:12.164652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd230c-959b-4cb7-8672-5418b6cb1b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:12.166238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aabd230c-959b-4cb7-8672-5418b6cb1b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:12.227539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:12.273991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd230c-959b-4cb7-8672-5418b6cb1b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:12.275439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd230c-959b-4cb7-8672-5418b6cb1b39 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:12.292744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 669/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:14.872136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554a79f-eeea-4df8-b99e-d10d761763ad None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:14.872556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554a79f-eeea-4df8-b99e-d10d761763ad None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:14.872702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554a79f-eeea-4df8-b99e-d10d761763ad None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:14.874673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 649/1318] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:14.879971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c1415-1e99-443a-986c-e46944c0cf5e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:14.880392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c1415-1e99-443a-986c-e46944c0cf5e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:14.880522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c1415-1e99-443a-986c-e46944c0cf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:14.945279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3e3c1415-1e99-443a-986c-e46944c0cf5e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:14.946481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3e3c1415-1e99-443a-986c-e46944c0cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:15.039015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 670/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:15.078268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554a79f-eeea-4df8-b99e-d10d761763ad None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:15.079839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d554a79f-eeea-4df8-b99e-d10d761763ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:15.180305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554a79f-eeea-4df8-b99e-d10d761763ad None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:15.250082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7febe-9379-4e54-b7af-d8395431dc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:15.250690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7febe-9379-4e54-b7af-d8395431dc7f service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:15.251001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7febe-9379-4e54-b7af-d8395431dc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:15.251277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7febe-9379-4e54-b7af-d8395431dc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:15.252863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-41c7febe-9379-4e54-b7af-d8395431dc7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:15.259958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:15.358778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7febe-9379-4e54-b7af-d8395431dc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:15.360234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7febe-9379-4e54-b7af-d8395431dc7f service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:15.378854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 650/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:18.039647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605cdc-9f06-47c7-b96f-b393e55f9952 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:18.040175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605cdc-9f06-47c7-b96f-b393e55f9952 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:18.040175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605cdc-9f06-47c7-b96f-b393e55f9952 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:18.041542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 671/1321] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:18.047232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbec26-4a96-407f-9413-5a492db25ba7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:18.047644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbec26-4a96-407f-9413-5a492db25ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:18.047820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbec26-4a96-407f-9413-5a492db25ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:18.112062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-09bbec26-4a96-407f-9413-5a492db25ba7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:18.113183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-09bbec26-4a96-407f-9413-5a492db25ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:18.169076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:18.213582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 651/1322] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:18.251142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b605cdc-9f06-47c7-b96f-b393e55f9952 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:18.252688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6b605cdc-9f06-47c7-b96f-b393e55f9952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:18.272908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:18.364258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b605cdc-9f06-47c7-b96f-b393e55f9952 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:18.437114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0949b2b-87e7-4109-9ea7-b053eaa235a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:18.437735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949b2b-87e7-4109-9ea7-b053eaa235a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:18.438016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949b2b-87e7-4109-9ea7-b053eaa235a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:18.438290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949b2b-87e7-4109-9ea7-b053eaa235a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:18.439960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c0949b2b-87e7-4109-9ea7-b053eaa235a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:18.513171 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:18.551521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949b2b-87e7-4109-9ea7-b053eaa235a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:18.552946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949b2b-87e7-4109-9ea7-b053eaa235a7 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:18.572051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 672/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:20.623203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201001d-93e7-4613-b958-0f707ca53253 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:20.623708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201001d-93e7-4613-b958-0f707ca53253 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:20.623708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201001d-93e7-4613-b958-0f707ca53253 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:20.625019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 652/1324] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:20.630146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502de9a-a9a5-4ad8-9f38-97698a616d5c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:20.630555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502de9a-a9a5-4ad8-9f38-97698a616d5c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:20.630844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502de9a-a9a5-4ad8-9f38-97698a616d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:20.697677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d502de9a-a9a5-4ad8-9f38-97698a616d5c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:20.698925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d502de9a-a9a5-4ad8-9f38-97698a616d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:20.795032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 673/1325] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:20.831572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201001d-93e7-4613-b958-0f707ca53253 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:20.833169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b201001d-93e7-4613-b958-0f707ca53253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:20.939466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201001d-93e7-4613-b958-0f707ca53253 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:21.012036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c22fcf-d22d-45d8-8ffa-2a46d36c7eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:21.012663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22fcf-d22d-45d8-8ffa-2a46d36c7eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:21.012940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22fcf-d22d-45d8-8ffa-2a46d36c7eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:21.013213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22fcf-d22d-45d8-8ffa-2a46d36c7eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:21.015479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a7c22fcf-d22d-45d8-8ffa-2a46d36c7eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:21.034832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:21.128641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c22fcf-d22d-45d8-8ffa-2a46d36c7eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:21.130087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c22fcf-d22d-45d8-8ffa-2a46d36c7eb4 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:21.148733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 653/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:23.219092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c6d2-cfa6-48a3-9640-de69f52b2354 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:23.219634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c6d2-cfa6-48a3-9640-de69f52b2354 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:23.219634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c6d2-cfa6-48a3-9640-de69f52b2354 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:23.220994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 674/1327] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:23.226129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cd875-08ed-4ee8-841f-a08b758ae325 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:23.226569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cd875-08ed-4ee8-841f-a08b758ae325 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:23.226682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cd875-08ed-4ee8-841f-a08b758ae325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:23.291379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-497cd875-08ed-4ee8-841f-a08b758ae325 None None] MFA Rules not processed for user `303ba9e42896448ca649a81fa5154966`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:23.292581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-497cd875-08ed-4ee8-841f-a08b758ae325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:23.358246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:23.391652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 654/1328] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:23.430413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c6d2-cfa6-48a3-9640-de69f52b2354 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:23.432100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3479c6d2-cfa6-48a3-9640-de69f52b2354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:23.465590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:23.538889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c6d2-cfa6-48a3-9640-de69f52b2354 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:23.615651 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10c986-d5b5-4a4e-997b-8c9bdbd90ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:23.616266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10c986-d5b5-4a4e-997b-8c9bdbd90ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:23.616544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10c986-d5b5-4a4e-997b-8c9bdbd90ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:23.616839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10c986-d5b5-4a4e-997b-8c9bdbd90ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:23.618433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7c10c986-d5b5-4a4e-997b-8c9bdbd90ed0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:23.697273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:23.729415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10c986-d5b5-4a4e-997b-8c9bdbd90ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:23.730850 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10c986-d5b5-4a4e-997b-8c9bdbd90ed0 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:23.750599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 675/1329] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:26.003067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42cea-0acb-43ae-bec7-7bcb5d46b160 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:26.003578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42cea-0acb-43ae-bec7-7bcb5d46b160 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:26.003578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42cea-0acb-43ae-bec7-7bcb5d46b160 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:26.004910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 655/1330] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:26.009952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc013f-c34f-45aa-9beb-23aea1ac642d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:26.010389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc013f-c34f-45aa-9beb-23aea1ac642d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:26.010485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc013f-c34f-45aa-9beb-23aea1ac642d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:26.075088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fddc013f-c34f-45aa-9beb-23aea1ac642d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:26.076275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fddc013f-c34f-45aa-9beb-23aea1ac642d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:26.169141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 676/1331] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:26.205763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42cea-0acb-43ae-bec7-7bcb5d46b160 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:26.207325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-35c42cea-0acb-43ae-bec7-7bcb5d46b160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:26.308419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42cea-0acb-43ae-bec7-7bcb5d46b160 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:26.383907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa573589-ac50-4c82-858c-0337fedeec80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:26.384522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa573589-ac50-4c82-858c-0337fedeec80 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:26.384802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa573589-ac50-4c82-858c-0337fedeec80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:26.385070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa573589-ac50-4c82-858c-0337fedeec80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:26.386668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-aa573589-ac50-4c82-858c-0337fedeec80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:26.412757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:26.491718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa573589-ac50-4c82-858c-0337fedeec80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:26.493140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa573589-ac50-4c82-858c-0337fedeec80 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:26.511567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 656/1332] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:28.553120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ca576-a1ea-47dc-a8f0-659656523ffc None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:28.553658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ca576-a1ea-47dc-a8f0-659656523ffc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:28.553658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ca576-a1ea-47dc-a8f0-659656523ffc None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:28.555051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 677/1333] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:28.560279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d19802-3b72-48a0-b0ea-e3a3f84a355c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:28.560687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d19802-3b72-48a0-b0ea-e3a3f84a355c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:28.560829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d19802-3b72-48a0-b0ea-e3a3f84a355c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:28.625807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-37d19802-3b72-48a0-b0ea-e3a3f84a355c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:28.627042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-37d19802-3b72-48a0-b0ea-e3a3f84a355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:28.698634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:28.721774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 657/1334] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:28.762886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ca576-a1ea-47dc-a8f0-659656523ffc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:28.765023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-684ca576-a1ea-47dc-a8f0-659656523ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:28.803840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:28.867517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ca576-a1ea-47dc-a8f0-659656523ffc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:28.940314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98bfe6-eb68-4ce0-b00f-52740645c781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:28.940934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98bfe6-eb68-4ce0-b00f-52740645c781 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:28.941217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98bfe6-eb68-4ce0-b00f-52740645c781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:28.941495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98bfe6-eb68-4ce0-b00f-52740645c781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:28.943136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fe98bfe6-eb68-4ce0-b00f-52740645c781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:29.024130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:29.050028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98bfe6-eb68-4ce0-b00f-52740645c781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:29.051744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98bfe6-eb68-4ce0-b00f-52740645c781 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:29.069941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 678/1335] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:31.405096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf855b37-0cd7-44ff-8e9a-a76bee602db9 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:31.405611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf855b37-0cd7-44ff-8e9a-a76bee602db9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:31.405732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf855b37-0cd7-44ff-8e9a-a76bee602db9 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:31.407028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 658/1336] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:31.412101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343e8d3-5630-4d23-886b-ee54afd2fd69 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:31.412498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343e8d3-5630-4d23-886b-ee54afd2fd69 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:31.412655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343e8d3-5630-4d23-886b-ee54afd2fd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:31.478743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0343e8d3-5630-4d23-886b-ee54afd2fd69 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:31.479971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0343e8d3-5630-4d23-886b-ee54afd2fd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:31.578298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 679/1337] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:31.630266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f9dc7-543b-4971-bddd-bbe62a0869d4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:31.630548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f9dc7-543b-4971-bddd-bbe62a0869d4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:31.630817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f9dc7-543b-4971-bddd-bbe62a0869d4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:31.632229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 659/1338] 10.222.0.17 () {54 vars in 965 bytes} [Sat Aug 6 14:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:31.642013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c187e-459d-4607-a69e-1a38d9b4c968 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:31.642332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c187e-459d-4607-a69e-1a38d9b4c968 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:31.642610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c187e-459d-4607-a69e-1a38d9b4c968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:31.695888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:31.717815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4a4c187e-459d-4607-a69e-1a38d9b4c968 None None] MFA Rules not processed for user `cd235744a7ee462eb13e383930de4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:31.719043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4a4c187e-459d-4607-a69e-1a38d9b4c968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:31.793376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 680/1339] 10.222.0.17 () {58 vars in 1062 bytes} [Sat Aug 6 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:31.802616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f9dc7-543b-4971-bddd-bbe62a0869d4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:31.804183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fc3f9dc7-543b-4971-bddd-bbe62a0869d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:31.904797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f9dc7-543b-4971-bddd-bbe62a0869d4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:31.975405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8c9f8-94e2-4b5c-8037-e9c1b1d382a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:31.975958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c9f8-94e2-4b5c-8037-e9c1b1d382a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:31.976242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c9f8-94e2-4b5c-8037-e9c1b1d382a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:31.976520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c9f8-94e2-4b5c-8037-e9c1b1d382a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:31.978203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f1c8c9f8-94e2-4b5c-8037-e9c1b1d382a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:32.022652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:32.094658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8c9f8-94e2-4b5c-8037-e9c1b1d382a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:32.096147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8c9f8-94e2-4b5c-8037-e9c1b1d382a4 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:32.114884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 660/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:36.578428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688776d2-b552-4f0c-8253-b705cf741c4b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:36.579046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688776d2-b552-4f0c-8253-b705cf741c4b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:36.579046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688776d2-b552-4f0c-8253-b705cf741c4b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:36.580589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 681/1341] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:36.585569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cb021-473c-4cf4-8fac-5645f287314a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:36.585998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cb021-473c-4cf4-8fac-5645f287314a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:36.586119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cb021-473c-4cf4-8fac-5645f287314a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:36.651153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-076cb021-473c-4cf4-8fac-5645f287314a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:36.652273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-076cb021-473c-4cf4-8fac-5645f287314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:36.737187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:36.750273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 661/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:36.788627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688776d2-b552-4f0c-8253-b705cf741c4b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:36.790190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-688776d2-b552-4f0c-8253-b705cf741c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:36.897862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688776d2-b552-4f0c-8253-b705cf741c4b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:36.937355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:36.983068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06367a76-1fe9-4bba-8f4d-568327aaf5e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:36.983775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367a76-1fe9-4bba-8f4d-568327aaf5e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:36.984058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367a76-1fe9-4bba-8f4d-568327aaf5e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:36.984330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367a76-1fe9-4bba-8f4d-568327aaf5e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:36.986026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-06367a76-1fe9-4bba-8f4d-568327aaf5e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:37.095596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06367a76-1fe9-4bba-8f4d-568327aaf5e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:37.097011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06367a76-1fe9-4bba-8f4d-568327aaf5e9 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:37.115541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 682/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:39.784876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c1ea0-9d76-4850-b724-06aab236a33f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:39.785408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c1ea0-9d76-4850-b724-06aab236a33f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:39.785408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c1ea0-9d76-4850-b724-06aab236a33f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:39.786893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 662/1344] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:39.792499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5642aa-e76a-4c7e-ab81-054a492929b6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:39.793077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5642aa-e76a-4c7e-ab81-054a492929b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:39.793077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5642aa-e76a-4c7e-ab81-054a492929b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:39.863245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4e5642aa-e76a-4c7e-ab81-054a492929b6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:39.864494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4e5642aa-e76a-4c7e-ab81-054a492929b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:39.883811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:39.962680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 683/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:22:40.000255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c1ea0-9d76-4850-b724-06aab236a33f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:40.001806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-307c1ea0-9d76-4850-b724-06aab236a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:40.102632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c1ea0-9d76-4850-b724-06aab236a33f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:40.175149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e4456-8f7d-4e6f-80c1-db2231ad7ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:40.176022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e4456-8f7d-4e6f-80c1-db2231ad7ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:40.176300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e4456-8f7d-4e6f-80c1-db2231ad7ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:40.176571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e4456-8f7d-4e6f-80c1-db2231ad7ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:40.178187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fb9e4456-8f7d-4e6f-80c1-db2231ad7ba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:40.220545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:40.285303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e4456-8f7d-4e6f-80c1-db2231ad7ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:40.286742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e4456-8f7d-4e6f-80c1-db2231ad7ba7 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:40.304888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 663/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:44.581956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43912-e679-4e18-8739-52ff8a794a81 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:44.582507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43912-e679-4e18-8739-52ff8a794a81 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:44.582507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43912-e679-4e18-8739-52ff8a794a81 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:44.583917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 684/1347] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:44.589433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25da0b-c156-45bc-9ded-df743652013b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:44.589921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25da0b-c156-45bc-9ded-df743652013b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:44.590036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25da0b-c156-45bc-9ded-df743652013b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:44.659026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-fc25da0b-c156-45bc-9ded-df743652013b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:44.660254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fc25da0b-c156-45bc-9ded-df743652013b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:44.760637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:44.763935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 664/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:44.802486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43912-e679-4e18-8739-52ff8a794a81 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:22:44.804089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1ed43912-e679-4e18-8739-52ff8a794a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:44.918457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43912-e679-4e18-8739-52ff8a794a81 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:22:44.969260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:44.997075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7dca7-5c26-47b0-97d8-490d61989998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:22:44.997742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7dca7-5c26-47b0-97d8-490d61989998 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:44.998017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7dca7-5c26-47b0-97d8-490d61989998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:44.998314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7dca7-5c26-47b0-97d8-490d61989998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:44.999974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-89e7dca7-5c26-47b0-97d8-490d61989998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:45.109474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7dca7-5c26-47b0-97d8-490d61989998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:22:45.110918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7dca7-5c26-47b0-97d8-490d61989998 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:22:45.129319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 685/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:22:50.005911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba454b0-f71d-4662-94f0-296c695adff6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:50.006336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba454b0-f71d-4662-94f0-296c695adff6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:50.006456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba454b0-f71d-4662-94f0-296c695adff6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:50.007839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 665/1350] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:50.013538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d083b0-3754-4162-8908-22e8e667a562 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:50.013957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d083b0-3754-4162-8908-22e8e667a562 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:50.014089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d083b0-3754-4162-8908-22e8e667a562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:50.114010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-92d083b0-3754-4162-8908-22e8e667a562 None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:50.115401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-92d083b0-3754-4162-8908-22e8e667a562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:50.146151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:50.259299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 686/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:54.460163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb641-52f1-48c7-8b19-0bece6fb1f25 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:54.461006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb641-52f1-48c7-8b19-0bece6fb1f25 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:54.461295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb641-52f1-48c7-8b19-0bece6fb1f25 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:54.462814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 666/1352] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:54.468279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f0fd-0982-4bdc-899b-71e430c618ea None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:54.468690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f0fd-0982-4bdc-899b-71e430c618ea None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:54.468855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f0fd-0982-4bdc-899b-71e430c618ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:54.547528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-74b7f0fd-0982-4bdc-899b-71e430c618ea None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:54.549018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-74b7f0fd-0982-4bdc-899b-71e430c618ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:54.625832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:54.692526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 687/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:22:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:55.900498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85c93e-8f09-41dc-83e8-c6196f531345 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:55.901056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85c93e-8f09-41dc-83e8-c6196f531345 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:55.901056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85c93e-8f09-41dc-83e8-c6196f531345 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:55.902390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 667/1354] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:55.907766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a8d1-df91-43e7-815d-7b88439395c0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:55.908172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a8d1-df91-43e7-815d-7b88439395c0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:55.908464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a8d1-df91-43e7-815d-7b88439395c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:55.985163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c975a8d1-df91-43e7-815d-7b88439395c0 None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:55.986359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c975a8d1-df91-43e7-815d-7b88439395c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:56.094810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 688/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:22:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:59.549015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8c2b-cfc0-4e9f-9464-6cdc89a78ab6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:59.549942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8c2b-cfc0-4e9f-9464-6cdc89a78ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:59.550672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8c2b-cfc0-4e9f-9464-6cdc89a78ab6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:59.556917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 668/1356] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:22:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:59.559080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4bdd4-c1ad-4020-aedb-c84514086c9a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:59.560029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4bdd4-c1ad-4020-aedb-c84514086c9a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:59.560511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4bdd4-c1ad-4020-aedb-c84514086c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:59.610784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:22:59.647557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e2d4bdd4-c1ad-4020-aedb-c84514086c9a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:59.649317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e2d4bdd4-c1ad-4020-aedb-c84514086c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:59.755266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 689/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:22:59.807537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad552c6-3530-4910-b48f-900762b5510b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:59.808174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad552c6-3530-4910-b48f-900762b5510b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:59.808730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad552c6-3530-4910-b48f-900762b5510b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:59.811112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 669/1358] 10.222.0.17 () {54 vars in 962 bytes} [Sat Aug 6 14:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:22:59.819837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569fea9-3d02-4f68-8771-3495cbaf4158 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:22:59.820710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569fea9-3d02-4f68-8771-3495cbaf4158 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:22:59.821216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569fea9-3d02-4f68-8771-3495cbaf4158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:22:59.910329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b569fea9-3d02-4f68-8771-3495cbaf4158 None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:22:59.911555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b569fea9-3d02-4f68-8771-3495cbaf4158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:22:59.964723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:00.038458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 690/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Aug 6 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:23:00.046667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad552c6-3530-4910-b48f-900762b5510b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:00.048318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5ad552c6-3530-4910-b48f-900762b5510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:00.199426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad552c6-3530-4910-b48f-900762b5510b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:00.276685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:00.322583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac5ebe-8933-4d43-bc96-24e5509504e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:00.323236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5ebe-8933-4d43-bc96-24e5509504e8 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:00.323550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5ebe-8933-4d43-bc96-24e5509504e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:00.323841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5ebe-8933-4d43-bc96-24e5509504e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:00.326024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e9ac5ebe-8933-4d43-bc96-24e5509504e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:00.417824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac5ebe-8933-4d43-bc96-24e5509504e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:00.419513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac5ebe-8933-4d43-bc96-24e5509504e8 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:00.439113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 670/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:23:02.672849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822b8c-5ac6-4eb6-9f51-a4af261ebece None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:02.673263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822b8c-5ac6-4eb6-9f51-a4af261ebece None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:02.673586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822b8c-5ac6-4eb6-9f51-a4af261ebece None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:02.674670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 691/1361] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:02.680542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5acca-9ae2-4c96-afcf-3cd1034d5372 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:02.681034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5acca-9ae2-4c96-afcf-3cd1034d5372 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:02.686512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5acca-9ae2-4c96-afcf-3cd1034d5372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:02.762008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-08f5acca-9ae2-4c96-afcf-3cd1034d5372 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:02.764179 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-08f5acca-9ae2-4c96-afcf-3cd1034d5372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:02.816096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:02.887679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 671/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:02.933649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822b8c-5ac6-4eb6-9f51-a4af261ebece None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:02.935275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-02822b8c-5ac6-4eb6-9f51-a4af261ebece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:03.118534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822b8c-5ac6-4eb6-9f51-a4af261ebece None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:03.129962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:03.259240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c234b2e-ef38-4db7-872c-b45033c99a85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:03.259922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c234b2e-ef38-4db7-872c-b45033c99a85 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:03.260142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c234b2e-ef38-4db7-872c-b45033c99a85 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:03.260420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c234b2e-ef38-4db7-872c-b45033c99a85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:03.262039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5c234b2e-ef38-4db7-872c-b45033c99a85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:03.364257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c234b2e-ef38-4db7-872c-b45033c99a85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:03.366104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c234b2e-ef38-4db7-872c-b45033c99a85 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:03.385856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 692/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:05.756570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404c9b5-55b7-4a3e-b562-f8ef78111462 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:05.757146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404c9b5-55b7-4a3e-b562-f8ef78111462 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:05.757146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404c9b5-55b7-4a3e-b562-f8ef78111462 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:05.758417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 672/1364] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:05.764971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa66cbd-f26d-43a9-afa1-9ad89119e681 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:05.765491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa66cbd-f26d-43a9-afa1-9ad89119e681 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:05.766068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa66cbd-f26d-43a9-afa1-9ad89119e681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:05.792079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:05.855619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2aa66cbd-f26d-43a9-afa1-9ad89119e681 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:05.856991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2aa66cbd-f26d-43a9-afa1-9ad89119e681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:05.962550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 693/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:06.011718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e912c1-7e0e-45e0-aa1f-7e77c691742e None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:06.011960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e912c1-7e0e-45e0-aa1f-7e77c691742e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:06.012487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e912c1-7e0e-45e0-aa1f-7e77c691742e None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:06.014140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 673/1366] 10.222.0.17 () {54 vars in 962 bytes} [Sat Aug 6 14:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:06.019708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68352-b1ad-40ed-9fcb-b1469f7a920d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:06.020001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68352-b1ad-40ed-9fcb-b1469f7a920d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:06.020282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68352-b1ad-40ed-9fcb-b1469f7a920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:06.106839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-18a68352-b1ad-40ed-9fcb-b1469f7a920d None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:06.108086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-18a68352-b1ad-40ed-9fcb-b1469f7a920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:06.144381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:06.233458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 694/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Aug 6 14:23:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:06.241763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e912c1-7e0e-45e0-aa1f-7e77c691742e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:06.243370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-54e912c1-7e0e-45e0-aa1f-7e77c691742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:06.407167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e912c1-7e0e-45e0-aa1f-7e77c691742e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:06.428878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:06.533270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475db8fc-a688-4190-83ce-8898c49c898f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:06.533902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db8fc-a688-4190-83ce-8898c49c898f service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:06.534179 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db8fc-a688-4190-83ce-8898c49c898f service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:06.534460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db8fc-a688-4190-83ce-8898c49c898f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:06.536147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-475db8fc-a688-4190-83ce-8898c49c898f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:06.629677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db8fc-a688-4190-83ce-8898c49c898f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:06.631476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db8fc-a688-4190-83ce-8898c49c898f service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:06.650851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 674/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:07.620532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e14cad-6c98-48d1-8a22-6a00ae5b5950 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:07.621099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e14cad-6c98-48d1-8a22-6a00ae5b5950 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:07.621391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e14cad-6c98-48d1-8a22-6a00ae5b5950 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:07.623790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 695/1369] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:07.629330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0752f9e-f3b6-4efd-a800-f57909d088df None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:07.629817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0752f9e-f3b6-4efd-a800-f57909d088df None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:07.629929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0752f9e-f3b6-4efd-a800-f57909d088df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:07.664244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:07.701758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a0752f9e-f3b6-4efd-a800-f57909d088df None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:07.702890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a0752f9e-f3b6-4efd-a800-f57909d088df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:07.845577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 675/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:08.198022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72003642-7d50-4b0e-975c-8c9a92482169 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:08.198381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72003642-7d50-4b0e-975c-8c9a92482169 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:08.198723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72003642-7d50-4b0e-975c-8c9a92482169 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:08.200878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 696/1371] 10.222.0.17 () {54 vars in 964 bytes} [Sat Aug 6 14:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:08.205759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a184ac-0098-431c-88bf-58ae005602f6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:08.206137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a184ac-0098-431c-88bf-58ae005602f6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:08.207683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a184ac-0098-431c-88bf-58ae005602f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:08.287077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c5a184ac-0098-431c-88bf-58ae005602f6 None None] MFA Rules not processed for user `8722d97d3ddc49a082ef84060a30d1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:08.288294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c5a184ac-0098-431c-88bf-58ae005602f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:08.336127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:08.368551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 676/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Aug 6 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:08.376551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72003642-7d50-4b0e-975c-8c9a92482169 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:08.378058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-72003642-7d50-4b0e-975c-8c9a92482169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:08.493012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72003642-7d50-4b0e-975c-8c9a92482169 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:08.535170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:08.570853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1178e8-044d-495b-9dbd-79df60219694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:08.571501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1178e8-044d-495b-9dbd-79df60219694 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:08.571782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1178e8-044d-495b-9dbd-79df60219694 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:08.572059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1178e8-044d-495b-9dbd-79df60219694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:08.573667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ab1178e8-044d-495b-9dbd-79df60219694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:08.686647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1178e8-044d-495b-9dbd-79df60219694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:08.688139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1178e8-044d-495b-9dbd-79df60219694 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:08.707534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 697/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:08.784291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768726-70c3-4ac8-bef5-1f60a57ecfeb None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:08.784837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768726-70c3-4ac8-bef5-1f60a57ecfeb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:08.785104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768726-70c3-4ac8-bef5-1f60a57ecfeb None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:08.786636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 677/1374] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:08.792047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655ffe-80d5-440c-84d9-9df7775f9a47 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:08.792345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655ffe-80d5-440c-84d9-9df7775f9a47 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:08.792628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655ffe-80d5-440c-84d9-9df7775f9a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:08.873181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a2655ffe-80d5-440c-84d9-9df7775f9a47 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:08.874712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a2655ffe-80d5-440c-84d9-9df7775f9a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:08.899967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:08.997986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 698/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:09.045673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09768726-70c3-4ac8-bef5-1f60a57ecfeb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:09.048533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-09768726-70c3-4ac8-bef5-1f60a57ecfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:09.201959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09768726-70c3-4ac8-bef5-1f60a57ecfeb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:09.252129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:09.313684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb3b3f-ed79-4757-9fa9-cc6110c22fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:09.314308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb3b3f-ed79-4757-9fa9-cc6110c22fb0 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:09.314620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb3b3f-ed79-4757-9fa9-cc6110c22fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:09.314914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb3b3f-ed79-4757-9fa9-cc6110c22fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:09.316553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-82bb3b3f-ed79-4757-9fa9-cc6110c22fb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:09.408949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb3b3f-ed79-4757-9fa9-cc6110c22fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:09.410634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb3b3f-ed79-4757-9fa9-cc6110c22fb0 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:09.429071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 678/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:11.459714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2037c-49cf-4846-b938-fe0f46f84b3d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:11.460248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2037c-49cf-4846-b938-fe0f46f84b3d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:11.460248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2037c-49cf-4846-b938-fe0f46f84b3d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:11.461622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 699/1377] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:11.467381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb88576-0f4c-485b-8c57-dff608223e31 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:11.467838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb88576-0f4c-485b-8c57-dff608223e31 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:11.467944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb88576-0f4c-485b-8c57-dff608223e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:11.535788 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dfb88576-0f4c-485b-8c57-dff608223e31 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:11.537005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dfb88576-0f4c-485b-8c57-dff608223e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:11.544183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:11.643774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 679/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:11.692508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2037c-49cf-4846-b938-fe0f46f84b3d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:11.694041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-38d2037c-49cf-4846-b938-fe0f46f84b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:11.853178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2037c-49cf-4846-b938-fe0f46f84b3d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:11.857336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:11.969076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd64e4-60b9-479a-bf09-4a5ece177f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:11.969735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd64e4-60b9-479a-bf09-4a5ece177f4b service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:11.970011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd64e4-60b9-479a-bf09-4a5ece177f4b service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:11.970280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd64e4-60b9-479a-bf09-4a5ece177f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:11.971904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b5bd64e4-60b9-479a-bf09-4a5ece177f4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:12.077701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd64e4-60b9-479a-bf09-4a5ece177f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:12.079419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd64e4-60b9-479a-bf09-4a5ece177f4b service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:12.101907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 700/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:14.163491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41608c-b706-4c5f-9dcd-bbdb0ebfbd31 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:14.164058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41608c-b706-4c5f-9dcd-bbdb0ebfbd31 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:14.164058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41608c-b706-4c5f-9dcd-bbdb0ebfbd31 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:14.166671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 680/1380] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:14.171501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12292c-dad0-4659-a1ce-e3365527e354 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:14.172082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12292c-dad0-4659-a1ce-e3365527e354 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:14.172082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12292c-dad0-4659-a1ce-e3365527e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:14.185848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:14.249227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-be12292c-dad0-4659-a1ce-e3365527e354 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:14.250421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-be12292c-dad0-4659-a1ce-e3365527e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:14.354255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 701/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:14.397565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41608c-b706-4c5f-9dcd-bbdb0ebfbd31 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:14.399239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7d41608c-b706-4c5f-9dcd-bbdb0ebfbd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:14.567052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:14.578995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41608c-b706-4c5f-9dcd-bbdb0ebfbd31 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:14.701445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc43e4d-28f1-48a4-913a-11ae2407fd1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:14.702056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc43e4d-28f1-48a4-913a-11ae2407fd1c service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:14.702336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc43e4d-28f1-48a4-913a-11ae2407fd1c service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:14.702610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc43e4d-28f1-48a4-913a-11ae2407fd1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:14.704268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0dc43e4d-28f1-48a4-913a-11ae2407fd1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:14.798517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc43e4d-28f1-48a4-913a-11ae2407fd1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:14.800149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc43e4d-28f1-48a4-913a-11ae2407fd1c service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:14.811771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:14.821484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 681/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:16.868815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532cd11-f7c0-4ba9-992d-305cd0f84735 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:16.869369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532cd11-f7c0-4ba9-992d-305cd0f84735 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:16.869369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532cd11-f7c0-4ba9-992d-305cd0f84735 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:16.870719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 702/1383] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:16.876130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490bd0f-7410-4417-971c-caa3e70f7dca None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:16.876645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490bd0f-7410-4417-971c-caa3e70f7dca None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:16.876751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490bd0f-7410-4417-971c-caa3e70f7dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:16.943149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b490bd0f-7410-4417-971c-caa3e70f7dca None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:16.944348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b490bd0f-7410-4417-971c-caa3e70f7dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:17.041692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 682/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:23:17.084298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532cd11-f7c0-4ba9-992d-305cd0f84735 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:17.086555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a532cd11-f7c0-4ba9-992d-305cd0f84735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:17.177867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:17.240429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532cd11-f7c0-4ba9-992d-305cd0f84735 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:17.353475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a7abe-a29a-4bba-8eef-7e6dc09fdf6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:17.354139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a7abe-a29a-4bba-8eef-7e6dc09fdf6d service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:17.354471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a7abe-a29a-4bba-8eef-7e6dc09fdf6d service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:17.354746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a7abe-a29a-4bba-8eef-7e6dc09fdf6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:17.356955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ec0a7abe-a29a-4bba-8eef-7e6dc09fdf6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:17.407577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:17.454556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a7abe-a29a-4bba-8eef-7e6dc09fdf6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:17.456510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a7abe-a29a-4bba-8eef-7e6dc09fdf6d service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:17.477820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 703/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:19.540028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90df3-0aec-4e74-8f14-72f34884b863 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:19.540581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90df3-0aec-4e74-8f14-72f34884b863 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:19.540581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90df3-0aec-4e74-8f14-72f34884b863 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:19.541950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 683/1386] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:23:19.547284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deea95-9234-4d92-9cbf-5931498d9e3e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:19.547812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deea95-9234-4d92-9cbf-5931498d9e3e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:19.547990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deea95-9234-4d92-9cbf-5931498d9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:19.614525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-36deea95-9234-4d92-9cbf-5931498d9e3e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:19.615771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-36deea95-9234-4d92-9cbf-5931498d9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:19.731700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 704/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:19.775046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90df3-0aec-4e74-8f14-72f34884b863 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:19.776779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-89e90df3-0aec-4e74-8f14-72f34884b863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:19.859475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:19.951822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90df3-0aec-4e74-8f14-72f34884b863 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:20.077206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6790dfd-57e2-4518-9387-705740bd314d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:20.077740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790dfd-57e2-4518-9387-705740bd314d service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:20.078019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790dfd-57e2-4518-9387-705740bd314d service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:20.078287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790dfd-57e2-4518-9387-705740bd314d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:20.080059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c6790dfd-57e2-4518-9387-705740bd314d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:20.127763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:20.189001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6790dfd-57e2-4518-9387-705740bd314d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:20.190619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6790dfd-57e2-4518-9387-705740bd314d service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:20.212842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 684/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:22.276897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78715733-72b4-4fad-879e-d79a4d9ecf96 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:22.278072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78715733-72b4-4fad-879e-d79a4d9ecf96 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:22.278072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78715733-72b4-4fad-879e-d79a4d9ecf96 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:22.279481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 705/1389] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:22.284816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4c8fe-a3f7-4c7b-bb34-312df0970f03 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:22.285254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4c8fe-a3f7-4c7b-bb34-312df0970f03 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:22.285395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4c8fe-a3f7-4c7b-bb34-312df0970f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:22.350556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b2d4c8fe-a3f7-4c7b-bb34-312df0970f03 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:22.351793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b2d4c8fe-a3f7-4c7b-bb34-312df0970f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:22.448239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 685/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:22.495164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78715733-72b4-4fad-879e-d79a4d9ecf96 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:22.497400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-78715733-72b4-4fad-879e-d79a4d9ecf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:22.530432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:22.646603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78715733-72b4-4fad-879e-d79a4d9ecf96 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:22.747903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:22.760002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf5dd8-3b50-4ea6-a3bf-c7d7ecd884e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:22.760651 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf5dd8-3b50-4ea6-a3bf-c7d7ecd884e7 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:22.760931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf5dd8-3b50-4ea6-a3bf-c7d7ecd884e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:22.761206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf5dd8-3b50-4ea6-a3bf-c7d7ecd884e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:22.762800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b5cf5dd8-3b50-4ea6-a3bf-c7d7ecd884e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:22.858196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf5dd8-3b50-4ea6-a3bf-c7d7ecd884e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:22.859828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf5dd8-3b50-4ea6-a3bf-c7d7ecd884e7 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:22.877806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 706/1391] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:23.699410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d8598-1eb0-4c88-b256-f4fb163c9671 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:23.700020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d8598-1eb0-4c88-b256-f4fb163c9671 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:23.700020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d8598-1eb0-4c88-b256-f4fb163c9671 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:23.701327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 686/1392] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:23.706935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b8b9-b6c4-47f9-a6fa-0fbbbd638220 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:23.707353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b8b9-b6c4-47f9-a6fa-0fbbbd638220 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:23.707449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6b8b9-b6c4-47f9-a6fa-0fbbbd638220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:23.773257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d8a6b8b9-b6c4-47f9-a6fa-0fbbbd638220 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:23.774523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d8a6b8b9-b6c4-47f9-a6fa-0fbbbd638220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:23.838229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:23.878900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 707/1393] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:23.934687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee246a8e-95da-4df5-a53f-e2320ded223d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:23.935004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee246a8e-95da-4df5-a53f-e2320ded223d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:23.935277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee246a8e-95da-4df5-a53f-e2320ded223d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:23.936652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 687/1394] 10.222.0.17 () {54 vars in 966 bytes} [Sat Aug 6 14:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:23.941830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302046dc-fcb2-4d80-ba95-1dc6b64298db None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:23.942122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302046dc-fcb2-4d80-ba95-1dc6b64298db None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:23.942403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302046dc-fcb2-4d80-ba95-1dc6b64298db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:24.021800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-302046dc-fcb2-4d80-ba95-1dc6b64298db None None] MFA Rules not processed for user `cf386f112e1b49fdba24871681961e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:24.022945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-302046dc-fcb2-4d80-ba95-1dc6b64298db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:24.102502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 708/1395] 10.222.0.17 () {58 vars in 1063 bytes} [Sat Aug 6 14:23:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:23:24.110193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee246a8e-95da-4df5-a53f-e2320ded223d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:24.111820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ee246a8e-95da-4df5-a53f-e2320ded223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:24.126188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:24.219503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee246a8e-95da-4df5-a53f-e2320ded223d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:24.291920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80029a09-3612-431c-be6b-be4482f7d695 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:24.292461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029a09-3612-431c-be6b-be4482f7d695 service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:24.292737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029a09-3612-431c-be6b-be4482f7d695 service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:24.293107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029a09-3612-431c-be6b-be4482f7d695 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:24.294700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-80029a09-3612-431c-be6b-be4482f7d695 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:24.362109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:24.403594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80029a09-3612-431c-be6b-be4482f7d695 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:24.405065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80029a09-3612-431c-be6b-be4482f7d695 service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:24.423546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 688/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:24.937757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abbd4b-9232-45e1-8ccf-c14cb9b5da00 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:24.938288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abbd4b-9232-45e1-8ccf-c14cb9b5da00 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:24.938288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abbd4b-9232-45e1-8ccf-c14cb9b5da00 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:24.939688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 709/1397] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:24.944965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb96d7c-2490-481c-a4c5-1fcdd1ac70d3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:24.945398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb96d7c-2490-481c-a4c5-1fcdd1ac70d3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:24.945502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb96d7c-2490-481c-a4c5-1fcdd1ac70d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:25.012614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0bb96d7c-2490-481c-a4c5-1fcdd1ac70d3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:25.013861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0bb96d7c-2490-481c-a4c5-1fcdd1ac70d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:25.118850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 689/1398] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:25.167481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abbd4b-9232-45e1-8ccf-c14cb9b5da00 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:25.169616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a0abbd4b-9232-45e1-8ccf-c14cb9b5da00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:25.169741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2c30c-c1e5-44f7-bf10-6b262f415122 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:25.170006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2c30c-c1e5-44f7-bf10-6b262f415122 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:25.170320 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2c30c-c1e5-44f7-bf10-6b262f415122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:25.197645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:25.235385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:25.239451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-55d2c30c-c1e5-44f7-bf10-6b262f415122 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:25.240647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-55d2c30c-c1e5-44f7-bf10-6b262f415122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:25.329618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abbd4b-9232-45e1-8ccf-c14cb9b5da00 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:25.446970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 690/1399] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:25.449827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:25.450458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:25.450740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:25.451045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:25.452719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:25.478208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:25.500075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40279f-4bbe-4519-afe5-9c7124faeb8d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:25.500739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40279f-4bbe-4519-afe5-9c7124faeb8d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:25.500739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40279f-4bbe-4519-afe5-9c7124faeb8d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:25.502168 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 691/1400] 10.222.0.17 () {54 vars in 943 bytes} [Sat Aug 6 14:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:23:25.508530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622654-36bf-445b-930b-08577742b233 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:25.508807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622654-36bf-445b-930b-08577742b233 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:25.509078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622654-36bf-445b-930b-08577742b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:25.551061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:25.552646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:25.570642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 710/1401] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:25.586565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c2622654-36bf-445b-930b-08577742b233 None None] MFA Rules not processed for user `add21760f9ed4b82b942c5dd9a6ed096`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:25.587903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c2622654-36bf-445b-930b-08577742b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:25.651051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:25.703712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 692/1402] 10.222.0.17 () {58 vars in 1040 bytes} [Sat Aug 6 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:23:25.714003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:25.715342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:25.826735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d603c0c-2e14-4fb5-b43f-5936aa09a3ec service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:25.887313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:25.935814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ef9a7-3ace-4aac-94a0-f84187d92459 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:25.936431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ef9a7-3ace-4aac-94a0-f84187d92459 service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:25.936723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ef9a7-3ace-4aac-94a0-f84187d92459 service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:25.936995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ef9a7-3ace-4aac-94a0-f84187d92459 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:25.938657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cb5ef9a7-3ace-4aac-94a0-f84187d92459 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:26.022673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ef9a7-3ace-4aac-94a0-f84187d92459 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:26.024217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ef9a7-3ace-4aac-94a0-f84187d92459 service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:26.033171 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 711/1403] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:23:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:23:27.645200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5de720-95e1-400e-a789-f97b915a6f2d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:27.645760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5de720-95e1-400e-a789-f97b915a6f2d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:27.645760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5de720-95e1-400e-a789-f97b915a6f2d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:27.647412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 693/1404] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:27.653173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d663194-70e9-4c71-9596-a43a1d02bea4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:27.653614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d663194-70e9-4c71-9596-a43a1d02bea4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:27.653710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d663194-70e9-4c71-9596-a43a1d02bea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:27.729712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0d663194-70e9-4c71-9596-a43a1d02bea4 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:27.731794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0d663194-70e9-4c71-9596-a43a1d02bea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:27.784225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:27.852981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 712/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:27.903967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5de720-95e1-400e-a789-f97b915a6f2d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:27.905576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7a5de720-95e1-400e-a789-f97b915a6f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:28.064675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5de720-95e1-400e-a789-f97b915a6f2d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:28.119982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:28.223884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84d76f-2696-4fe8-ae03-adca7c7f2d0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:28.224541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d76f-2696-4fe8-ae03-adca7c7f2d0b service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:28.224823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d76f-2696-4fe8-ae03-adca7c7f2d0b service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:28.225096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84d76f-2696-4fe8-ae03-adca7c7f2d0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:28.226723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9f84d76f-2696-4fe8-ae03-adca7c7f2d0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:28.323583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84d76f-2696-4fe8-ae03-adca7c7f2d0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:28.325634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84d76f-2696-4fe8-ae03-adca7c7f2d0b service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:28.344903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 694/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:30.424347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0b33-be97-4faa-9b61-c7df2152d475 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:30.424881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0b33-be97-4faa-9b61-c7df2152d475 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:30.424881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0b33-be97-4faa-9b61-c7df2152d475 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:30.427298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 713/1407] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:23:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:30.431750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d28e7-4fff-484e-93fe-db46c71615f0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:30.432198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d28e7-4fff-484e-93fe-db46c71615f0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:30.432295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d28e7-4fff-484e-93fe-db46c71615f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:30.475088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:30.506742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-835d28e7-4fff-484e-93fe-db46c71615f0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:30.507992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-835d28e7-4fff-484e-93fe-db46c71615f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:30.604219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 695/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:23:30.647537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc0b33-be97-4faa-9b61-c7df2152d475 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:30.649062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1ebc0b33-be97-4faa-9b61-c7df2152d475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:30.815968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc0b33-be97-4faa-9b61-c7df2152d475 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:30.853714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:30.952427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c5458-c220-459e-bd6c-6c24c4558f46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:30.953165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c5458-c220-459e-bd6c-6c24c4558f46 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:30.953519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c5458-c220-459e-bd6c-6c24c4558f46 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:30.953869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c5458-c220-459e-bd6c-6c24c4558f46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:30.955624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f26c5458-c220-459e-bd6c-6c24c4558f46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:31.055116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c5458-c220-459e-bd6c-6c24c4558f46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:31.056931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c5458-c220-459e-bd6c-6c24c4558f46 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:31.076069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 714/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:40.602166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=51 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:41.073644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:48.481195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7e44c-3969-439c-a7d4-21983da7f297 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:48.481742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7e44c-3969-439c-a7d4-21983da7f297 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:48.481742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7e44c-3969-439c-a7d4-21983da7f297 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:48.483155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 696/1410] 10.222.0.17 () {54 vars in 918 bytes} [Sat Aug 6 14:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:48.488353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607188d-de64-496c-ae9c-1a6c9f2c5e22 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:48.488786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607188d-de64-496c-ae9c-1a6c9f2c5e22 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:48.488907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607188d-de64-496c-ae9c-1a6c9f2c5e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:48.563588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1607188d-de64-496c-ae9c-1a6c9f2c5e22 None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:48.564772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1607188d-de64-496c-ae9c-1a6c9f2c5e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:48.666605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 715/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Aug 6 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:48.681217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad51d1-ebd6-4238-9589-175fcee31439 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:48.681501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad51d1-ebd6-4238-9589-175fcee31439 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:48.681776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad51d1-ebd6-4238-9589-175fcee31439 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:48.683757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 697/1412] 10.222.0.17 () {54 vars in 918 bytes} [Sat Aug 6 14:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:48.688167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df84cd-5831-4b53-afd4-d6f96de3bc66 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:48.688452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df84cd-5831-4b53-afd4-d6f96de3bc66 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:48.688727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df84cd-5831-4b53-afd4-d6f96de3bc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:48.767477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:48.770474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-59df84cd-5831-4b53-afd4-d6f96de3bc66 None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:48.772316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-59df84cd-5831-4b53-afd4-d6f96de3bc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:48.888288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 716/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Aug 6 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:51.332369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7341bcf-284b-42a1-b1a7-5b370a6ab313 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:51.332908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7341bcf-284b-42a1-b1a7-5b370a6ab313 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:51.332908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7341bcf-284b-42a1-b1a7-5b370a6ab313 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:51.334252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:51.339657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e082-6a45-433a-9259-54030fd1fae2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:51.340058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e082-6a45-433a-9259-54030fd1fae2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:51.340229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e082-6a45-433a-9259-54030fd1fae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:51.407578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d826e082-6a45-433a-9259-54030fd1fae2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:51.408783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d826e082-6a45-433a-9259-54030fd1fae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:51.465821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:51.509932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:51.555866 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7341bcf-284b-42a1-b1a7-5b370a6ab313 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:51.557388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d7341bcf-284b-42a1-b1a7-5b370a6ab313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:51.699505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7341bcf-284b-42a1-b1a7-5b370a6ab313 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:51.770487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:51.810396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53905672-3661-418c-9071-a7d6b0d3c396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:51.811058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53905672-3661-418c-9071-a7d6b0d3c396 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:51.811341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53905672-3661-418c-9071-a7d6b0d3c396 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:51.811616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53905672-3661-418c-9071-a7d6b0d3c396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:51.813272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-53905672-3661-418c-9071-a7d6b0d3c396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:51.900304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53905672-3661-418c-9071-a7d6b0d3c396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:51.902403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53905672-3661-418c-9071-a7d6b0d3c396 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:51.920430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 699/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:52.812791 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bddc18-1412-404b-975a-1b86ed5db237 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:52.813333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bddc18-1412-404b-975a-1b86ed5db237 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:52.813333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bddc18-1412-404b-975a-1b86ed5db237 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:52.814728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:52.819857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee749-c29e-4055-9b8f-86f1f7d3b336 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:52.820253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee749-c29e-4055-9b8f-86f1f7d3b336 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:52.820399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee749-c29e-4055-9b8f-86f1f7d3b336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:52.894265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-93fee749-c29e-4055-9b8f-86f1f7d3b336 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:52.895469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-93fee749-c29e-4055-9b8f-86f1f7d3b336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:52.920981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:52.989266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:53.142981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bddc18-1412-404b-975a-1b86ed5db237 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:53.144491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-84bddc18-1412-404b-975a-1b86ed5db237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:53.287383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bddc18-1412-404b-975a-1b86ed5db237 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:53.327265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:53.401015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da689-1710-4db3-bf42-0d522b5ffc33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:53.401546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da689-1710-4db3-bf42-0d522b5ffc33 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:53.402318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da689-1710-4db3-bf42-0d522b5ffc33 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:53.402593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da689-1710-4db3-bf42-0d522b5ffc33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:53.404247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5f1da689-1710-4db3-bf42-0d522b5ffc33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:53.492723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1da689-1710-4db3-bf42-0d522b5ffc33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:53.494328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1da689-1710-4db3-bf42-0d522b5ffc33 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:53.512395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 719/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:54.380769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b7a3-4b4f-4f47-851f-38516b654afe None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:54.381390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b7a3-4b4f-4f47-851f-38516b654afe None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:54.381390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b7a3-4b4f-4f47-851f-38516b654afe None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:54.382667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:54.388190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c5ddc-044b-435a-b058-9b8ac07bef43 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:54.388587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c5ddc-044b-435a-b058-9b8ac07bef43 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:54.388732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c5ddc-044b-435a-b058-9b8ac07bef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:54.447879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:54.470590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e63c5ddc-044b-435a-b058-9b8ac07bef43 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:54.471846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e63c5ddc-044b-435a-b058-9b8ac07bef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:54.568040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:54.721977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754b7a3-4b4f-4f47-851f-38516b654afe None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:54.721977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6754b7a3-4b4f-4f47-851f-38516b654afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:54.866011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754b7a3-4b4f-4f47-851f-38516b654afe None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:54.878489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:54.977817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b1b54-4e6b-4374-9f03-bbe27447ad00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:54.978355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b1b54-4e6b-4374-9f03-bbe27447ad00 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:54.978635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b1b54-4e6b-4374-9f03-bbe27447ad00 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:54.978927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b1b54-4e6b-4374-9f03-bbe27447ad00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:54.981108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1c4b1b54-4e6b-4374-9f03-bbe27447ad00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:55.073881 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b1b54-4e6b-4374-9f03-bbe27447ad00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:55.075519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b1b54-4e6b-4374-9f03-bbe27447ad00 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:55.093446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 702/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:55.951665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa84cd-819d-4a4e-886c-fe4f08a0ac6a None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:55.952303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa84cd-819d-4a4e-886c-fe4f08a0ac6a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:55.952303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa84cd-819d-4a4e-886c-fe4f08a0ac6a None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:55.953712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:55.958739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321fc16-08fe-4008-9955-592de734ca76 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:55.959179 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321fc16-08fe-4008-9955-592de734ca76 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:55.959336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321fc16-08fe-4008-9955-592de734ca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:55.978466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:56.033401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8321fc16-08fe-4008-9955-592de734ca76 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:56.034599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8321fc16-08fe-4008-9955-592de734ca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:56.128298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:56.278301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa84cd-819d-4a4e-886c-fe4f08a0ac6a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:56.279862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-61fa84cd-819d-4a4e-886c-fe4f08a0ac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:56.395514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:56.432468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa84cd-819d-4a4e-886c-fe4f08a0ac6a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:56.544312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2246b1-e732-4040-8e1e-f8ac6af255a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:56.544919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2246b1-e732-4040-8e1e-f8ac6af255a3 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:56.545195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2246b1-e732-4040-8e1e-f8ac6af255a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:56.545466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2246b1-e732-4040-8e1e-f8ac6af255a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:56.547098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bd2246b1-e732-4040-8e1e-f8ac6af255a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:56.620290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:56.634093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2246b1-e732-4040-8e1e-f8ac6af255a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:56.635712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2246b1-e732-4040-8e1e-f8ac6af255a3 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:56.653755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 722/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:57.506517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f16b0-12d4-45e3-bd1e-52a35a6ceddd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:57.507085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f16b0-12d4-45e3-bd1e-52a35a6ceddd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:57.507205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f16b0-12d4-45e3-bd1e-52a35a6ceddd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:57.508760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:23:57.514479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca245-0b36-4853-9df0-287e7933401a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:57.514920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca245-0b36-4853-9df0-287e7933401a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:57.515054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca245-0b36-4853-9df0-287e7933401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:57.591164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c41ca245-0b36-4853-9df0-287e7933401a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:57.592351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c41ca245-0b36-4853-9df0-287e7933401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:57.686644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:23:57.838602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f16b0-12d4-45e3-bd1e-52a35a6ceddd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:57.840078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d72f16b0-12d4-45e3-bd1e-52a35a6ceddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:57.923469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:57.986286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f16b0-12d4-45e3-bd1e-52a35a6ceddd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:58.103272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971506e-0f24-4610-b57a-ca17d98256a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:58.103807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971506e-0f24-4610-b57a-ca17d98256a8 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:58.104084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971506e-0f24-4610-b57a-ca17d98256a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:58.104356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971506e-0f24-4610-b57a-ca17d98256a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:58.106563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c971506e-0f24-4610-b57a-ca17d98256a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:58.153451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:58.194329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971506e-0f24-4610-b57a-ca17d98256a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:58.195973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971506e-0f24-4610-b57a-ca17d98256a8 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:58.213616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 705/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:23:59.006687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cdf09-1e7d-4e2f-a2ac-e366ca74d913 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:59.007274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cdf09-1e7d-4e2f-a2ac-e366ca74d913 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:59.007274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cdf09-1e7d-4e2f-a2ac-e366ca74d913 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:59.008611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 14:23:59.014213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877683-fc0c-44dc-a860-5c75e04ae99d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:59.014620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877683-fc0c-44dc-a860-5c75e04ae99d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:59.014750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877683-fc0c-44dc-a860-5c75e04ae99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:59.087675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-02877683-fc0c-44dc-a860-5c75e04ae99d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:23:59.088855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-02877683-fc0c-44dc-a860-5c75e04ae99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:59.184080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:23:59.334425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cdf09-1e7d-4e2f-a2ac-e366ca74d913 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:23:59.335980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-809cdf09-1e7d-4e2f-a2ac-e366ca74d913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:59.384412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:59.480712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cdf09-1e7d-4e2f-a2ac-e366ca74d913 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:23:59.592337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b3aed-f792-447c-8ae3-dc7a2058fd86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:23:59.592948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b3aed-f792-447c-8ae3-dc7a2058fd86 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:23:59.593224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b3aed-f792-447c-8ae3-dc7a2058fd86 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:23:59.593491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b3aed-f792-447c-8ae3-dc7a2058fd86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:23:59.595116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6e2b3aed-f792-447c-8ae3-dc7a2058fd86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:23:59.601789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:23:59.689018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b3aed-f792-447c-8ae3-dc7a2058fd86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:23:59.690600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b3aed-f792-447c-8ae3-dc7a2058fd86 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:23:59.708679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 725/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:23:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:00.508892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87636ff-1465-4ad7-b445-3d2115a1fa1b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:00.509464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87636ff-1465-4ad7-b445-3d2115a1fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:00.509464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87636ff-1465-4ad7-b445-3d2115a1fa1b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:00.510837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:00.516519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53502de-c77d-4c00-99be-d08dcc01fcd7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:00.516959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53502de-c77d-4c00-99be-d08dcc01fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:00.517073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53502de-c77d-4c00-99be-d08dcc01fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:00.591926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a53502de-c77d-4c00-99be-d08dcc01fcd7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:00.593109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a53502de-c77d-4c00-99be-d08dcc01fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:00.659196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:00.687431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:24:00.838052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87636ff-1465-4ad7-b445-3d2115a1fa1b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:00.839588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a87636ff-1465-4ad7-b445-3d2115a1fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:00.857400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:00.981475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87636ff-1465-4ad7-b445-3d2115a1fa1b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:01.068292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:01.094409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b51f4-bdb0-40fb-9bfb-d8a72e2c6345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:01.094986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b51f4-bdb0-40fb-9bfb-d8a72e2c6345 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:01.095266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b51f4-bdb0-40fb-9bfb-d8a72e2c6345 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:01.095538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b51f4-bdb0-40fb-9bfb-d8a72e2c6345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:01.097149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f02b51f4-bdb0-40fb-9bfb-d8a72e2c6345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:01.183854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b51f4-bdb0-40fb-9bfb-d8a72e2c6345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:01.185447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b51f4-bdb0-40fb-9bfb-d8a72e2c6345 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:01.203549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 708/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:01.993350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4142c6-6863-4efc-a3e9-e8704dc87c1d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:01.993903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4142c6-6863-4efc-a3e9-e8704dc87c1d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:01.993903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4142c6-6863-4efc-a3e9-e8704dc87c1d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:01.995315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:02.000231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc636-2992-4608-b242-adb37263643b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:02.000640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc636-2992-4608-b242-adb37263643b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:02.000780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc636-2992-4608-b242-adb37263643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:02.074747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4b5dc636-2992-4608-b242-adb37263643b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:02.075969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4b5dc636-2992-4608-b242-adb37263643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:02.116117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:02.169765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:02.319968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4142c6-6863-4efc-a3e9-e8704dc87c1d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:02.321611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0a4142c6-6863-4efc-a3e9-e8704dc87c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:02.468364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4142c6-6863-4efc-a3e9-e8704dc87c1d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:02.526106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:02.585957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc8b26-0bc6-46c3-93fb-496f584e0801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:02.586583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc8b26-0bc6-46c3-93fb-496f584e0801 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:02.586936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc8b26-0bc6-46c3-93fb-496f584e0801 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:02.587163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc8b26-0bc6-46c3-93fb-496f584e0801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:02.588788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-77bc8b26-0bc6-46c3-93fb-496f584e0801 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:02.678342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc8b26-0bc6-46c3-93fb-496f584e0801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:02.680015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc8b26-0bc6-46c3-93fb-496f584e0801 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:02.702181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 728/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:03.507387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23638-25fb-4ae3-81eb-e9a02f8f7865 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:03.507887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23638-25fb-4ae3-81eb-e9a02f8f7865 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:03.508012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23638-25fb-4ae3-81eb-e9a02f8f7865 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:03.509289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:03.514899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f131-acc7-4f31-a415-33658fdca1b7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:03.515305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f131-acc7-4f31-a415-33658fdca1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:03.515466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f131-acc7-4f31-a415-33658fdca1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:03.590328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8e63f131-acc7-4f31-a415-33658fdca1b7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:03.591585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8e63f131-acc7-4f31-a415-33658fdca1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:03.593575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:03.687210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:03.841320 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee23638-25fb-4ae3-81eb-e9a02f8f7865 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:03.845155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bee23638-25fb-4ae3-81eb-e9a02f8f7865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:03.988791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee23638-25fb-4ae3-81eb-e9a02f8f7865 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:04.015353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:04.110141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa9c2-2b2d-4122-ae50-b6575b2d7c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:04.110668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa9c2-2b2d-4122-ae50-b6575b2d7c6f service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:04.110970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa9c2-2b2d-4122-ae50-b6575b2d7c6f service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:04.111246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa9c2-2b2d-4122-ae50-b6575b2d7c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:04.112847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e00fa9c2-2b2d-4122-ae50-b6575b2d7c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:04.198680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa9c2-2b2d-4122-ae50-b6575b2d7c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:04.200299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa9c2-2b2d-4122-ae50-b6575b2d7c6f service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:04.218221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 711/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:06.125379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080b21-d366-43b7-ba5b-b3cf47abc72c None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:06.125921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080b21-d366-43b7-ba5b-b3cf47abc72c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:06.125921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080b21-d366-43b7-ba5b-b3cf47abc72c None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:06.127329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:06.132512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687f133-4520-4755-abfa-ad4b587713ed None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:06.132943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687f133-4520-4755-abfa-ad4b587713ed None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:06.133039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687f133-4520-4755-abfa-ad4b587713ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:06.174997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:06.200011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7687f133-4520-4755-abfa-ad4b587713ed None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:06.201216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7687f133-4520-4755-abfa-ad4b587713ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:06.295192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:06.332990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080b21-d366-43b7-ba5b-b3cf47abc72c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:06.334468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f1080b21-d366-43b7-ba5b-b3cf47abc72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:06.435241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080b21-d366-43b7-ba5b-b3cf47abc72c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:06.466409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:06.507254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40d80f-0c4d-4738-9db2-3056e250a09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:06.507940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40d80f-0c4d-4738-9db2-3056e250a09b service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:06.508295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40d80f-0c4d-4738-9db2-3056e250a09b service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:06.508562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40d80f-0c4d-4738-9db2-3056e250a09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:06.510191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8a40d80f-0c4d-4738-9db2-3056e250a09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:06.616098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40d80f-0c4d-4738-9db2-3056e250a09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:06.617484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40d80f-0c4d-4738-9db2-3056e250a09b service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:06.634973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 731/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:08.798655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef754b9-09b5-45d8-8f55-8f610c788744 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:08.799237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef754b9-09b5-45d8-8f55-8f610c788744 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:08.799237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef754b9-09b5-45d8-8f55-8f610c788744 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:08.800600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:08.806039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaf822-67bb-4c29-acd6-579549f154b7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:08.806452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaf822-67bb-4c29-acd6-579549f154b7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:08.806593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaf822-67bb-4c29-acd6-579549f154b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:08.874005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-aeeaf822-67bb-4c29-acd6-579549f154b7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:08.875271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aeeaf822-67bb-4c29-acd6-579549f154b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:08.888131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:09.090392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:09.129237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef754b9-09b5-45d8-8f55-8f610c788744 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:09.130780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-eef754b9-09b5-45d8-8f55-8f610c788744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:09.231894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef754b9-09b5-45d8-8f55-8f610c788744 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:09.243675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:09.305118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34c5e5-3080-4f8f-b1e0-edeceeeb87c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:09.305732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34c5e5-3080-4f8f-b1e0-edeceeeb87c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:09.306008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34c5e5-3080-4f8f-b1e0-edeceeeb87c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:09.306273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34c5e5-3080-4f8f-b1e0-edeceeeb87c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:09.307917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2e34c5e5-3080-4f8f-b1e0-edeceeeb87c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:09.412205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34c5e5-3080-4f8f-b1e0-edeceeeb87c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:09.413618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34c5e5-3080-4f8f-b1e0-edeceeeb87c5 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:09.431671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 714/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:11.738293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e872b78-58de-47db-93b6-446c9389508e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:11.738806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e872b78-58de-47db-93b6-446c9389508e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:11.739062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e872b78-58de-47db-93b6-446c9389508e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:11.741098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:11.746382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710944d7-7197-4cec-b182-4ebb48bd2423 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:11.746862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710944d7-7197-4cec-b182-4ebb48bd2423 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:11.747056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710944d7-7197-4cec-b182-4ebb48bd2423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:11.795468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:11.813493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-710944d7-7197-4cec-b182-4ebb48bd2423 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:11.814709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-710944d7-7197-4cec-b182-4ebb48bd2423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:11.910242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:11.949398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e872b78-58de-47db-93b6-446c9389508e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:11.950926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0e872b78-58de-47db-93b6-446c9389508e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:12.053218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e872b78-58de-47db-93b6-446c9389508e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:12.092313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:12.126020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f87151-8d3b-4c69-976c-d26289b9639e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:12.126626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87151-8d3b-4c69-976c-d26289b9639e service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:12.126928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87151-8d3b-4c69-976c-d26289b9639e service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:12.127202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87151-8d3b-4c69-976c-d26289b9639e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:12.128788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-52f87151-8d3b-4c69-976c-d26289b9639e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:12.233269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f87151-8d3b-4c69-976c-d26289b9639e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:12.234668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f87151-8d3b-4c69-976c-d26289b9639e service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:12.252402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 734/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:14.375155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b735f04-f723-4852-9980-84fcb4ad6dc6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:14.375723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b735f04-f723-4852-9980-84fcb4ad6dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:14.375723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b735f04-f723-4852-9980-84fcb4ad6dc6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:14.377052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:14.382188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fd257-3309-4187-b508-97dba7ed4a17 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:14.382631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fd257-3309-4187-b508-97dba7ed4a17 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:14.382746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fd257-3309-4187-b508-97dba7ed4a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:14.447138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ed9fd257-3309-4187-b508-97dba7ed4a17 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:14.448363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ed9fd257-3309-4187-b508-97dba7ed4a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:14.473578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:14.552071 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:14.590817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b735f04-f723-4852-9980-84fcb4ad6dc6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:14.592401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7b735f04-f723-4852-9980-84fcb4ad6dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:14.696986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b735f04-f723-4852-9980-84fcb4ad6dc6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:14.717525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:14.770592 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139bec2-9586-4a46-8e48-0741337d5cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:14.771240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139bec2-9586-4a46-8e48-0741337d5cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:14.771521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139bec2-9586-4a46-8e48-0741337d5cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:14.771795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139bec2-9586-4a46-8e48-0741337d5cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:14.773407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e139bec2-9586-4a46-8e48-0741337d5cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:14.881426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139bec2-9586-4a46-8e48-0741337d5cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:14.882858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139bec2-9586-4a46-8e48-0741337d5cb8 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:14.903732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 717/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:17.003954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69618-62f4-4326-9a28-686edac5ed65 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:17.004484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69618-62f4-4326-9a28-686edac5ed65 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:17.004484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69618-62f4-4326-9a28-686edac5ed65 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:17.005855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:17.011550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f23c4-35f1-4517-b060-565e7d6a9eee None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:17.011986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f23c4-35f1-4517-b060-565e7d6a9eee None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:17.012096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f23c4-35f1-4517-b060-565e7d6a9eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:17.074052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:17.077172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7d5f23c4-35f1-4517-b060-565e7d6a9eee None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:17.078798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7d5f23c4-35f1-4517-b060-565e7d6a9eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:17.172246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:17.210581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa69618-62f4-4326-9a28-686edac5ed65 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:17.212161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1aa69618-62f4-4326-9a28-686edac5ed65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:17.314103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa69618-62f4-4326-9a28-686edac5ed65 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:17.361284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:17.385845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f547ff-166b-4bb9-b359-0f1341e87afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:17.386497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f547ff-166b-4bb9-b359-0f1341e87afc service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:17.386780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f547ff-166b-4bb9-b359-0f1341e87afc service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:17.387083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f547ff-166b-4bb9-b359-0f1341e87afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:17.388704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-33f547ff-166b-4bb9-b359-0f1341e87afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:17.494748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f547ff-166b-4bb9-b359-0f1341e87afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:17.496214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f547ff-166b-4bb9-b359-0f1341e87afc service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:17.514772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 737/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:19.551898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40a20a-4a21-4554-905c-77cd8a894356 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:19.552454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40a20a-4a21-4554-905c-77cd8a894356 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:19.552454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40a20a-4a21-4554-905c-77cd8a894356 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:19.553811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:19.559138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274cf1-e0bc-459a-af9b-59a234911ebb None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:19.559542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274cf1-e0bc-459a-af9b-59a234911ebb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:19.559692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274cf1-e0bc-459a-af9b-59a234911ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:19.626849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f3274cf1-e0bc-459a-af9b-59a234911ebb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:19.628064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f3274cf1-e0bc-459a-af9b-59a234911ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:19.656528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:19.724090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:19.763715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40a20a-4a21-4554-905c-77cd8a894356 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:19.765249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9f40a20a-4a21-4554-905c-77cd8a894356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:19.873584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40a20a-4a21-4554-905c-77cd8a894356 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:19.901548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:19.945654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e6033-a985-47f5-afc1-2d74a2982fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:19.946268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6033-a985-47f5-afc1-2d74a2982fae service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:19.946548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6033-a985-47f5-afc1-2d74a2982fae service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:19.946821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6033-a985-47f5-afc1-2d74a2982fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:19.949062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0e2e6033-a985-47f5-afc1-2d74a2982fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:20.060072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e6033-a985-47f5-afc1-2d74a2982fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:20.062175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e6033-a985-47f5-afc1-2d74a2982fae service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:20.086717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 720/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:22.110456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9e190-35c5-49c5-baf7-805db9a59c3b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:22.111019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9e190-35c5-49c5-baf7-805db9a59c3b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:22.111019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9e190-35c5-49c5-baf7-805db9a59c3b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:22.112321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:22.117427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155ca7-6cb1-4752-9fa0-d626b60238d2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:22.117848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155ca7-6cb1-4752-9fa0-d626b60238d2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:22.117964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155ca7-6cb1-4752-9fa0-d626b60238d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:22.182620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d3155ca7-6cb1-4752-9fa0-d626b60238d2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:22.183855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d3155ca7-6cb1-4752-9fa0-d626b60238d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:22.191940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:22.278545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:22.315582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9e190-35c5-49c5-baf7-805db9a59c3b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:22.317147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-45c9e190-35c5-49c5-baf7-805db9a59c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:22.418628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9e190-35c5-49c5-baf7-805db9a59c3b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:22.472583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:22.490653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739e6e4-53ed-44a7-9fc9-4dcc06c01003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:22.491297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739e6e4-53ed-44a7-9fc9-4dcc06c01003 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:22.491578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739e6e4-53ed-44a7-9fc9-4dcc06c01003 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:22.491871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739e6e4-53ed-44a7-9fc9-4dcc06c01003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:22.493458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e739e6e4-53ed-44a7-9fc9-4dcc06c01003 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:22.602943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739e6e4-53ed-44a7-9fc9-4dcc06c01003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:22.604391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739e6e4-53ed-44a7-9fc9-4dcc06c01003 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:22.622366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 740/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:24.662679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d38d-2806-453d-9a34-c39ae31656b1 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:24.663262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d38d-2806-453d-9a34-c39ae31656b1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:24.663418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d38d-2806-453d-9a34-c39ae31656b1 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:24.664802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:24.670649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea704e-6279-4f77-a587-5f6a8a640c8f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:24.671118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea704e-6279-4f77-a587-5f6a8a640c8f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:24.671248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea704e-6279-4f77-a587-5f6a8a640c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:24.736346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b8ea704e-6279-4f77-a587-5f6a8a640c8f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:24.737573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b8ea704e-6279-4f77-a587-5f6a8a640c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:24.775113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:24.833375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:24.871185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757d38d-2806-453d-9a34-c39ae31656b1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:24.872779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8757d38d-2806-453d-9a34-c39ae31656b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:24.974032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757d38d-2806-453d-9a34-c39ae31656b1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:25.010493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:25.047460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc7e65-1485-460a-af0c-502c9fe8b911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:25.048083 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7e65-1485-460a-af0c-502c9fe8b911 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:25.048367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7e65-1485-460a-af0c-502c9fe8b911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:25.048649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7e65-1485-460a-af0c-502c9fe8b911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:25.050271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9fbc7e65-1485-460a-af0c-502c9fe8b911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:25.166005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc7e65-1485-460a-af0c-502c9fe8b911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:25.168056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc7e65-1485-460a-af0c-502c9fe8b911 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:25.186218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 723/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:27.228716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec32a68-791d-4366-9a48-54385404e9d8 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:27.229873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec32a68-791d-4366-9a48-54385404e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:27.229873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec32a68-791d-4366-9a48-54385404e9d8 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:27.231235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:27.236337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c526e-a27e-42b8-a76d-6d0421dca529 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:27.236778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c526e-a27e-42b8-a76d-6d0421dca529 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:27.236875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c526e-a27e-42b8-a76d-6d0421dca529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:27.302466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-642c526e-a27e-42b8-a76d-6d0421dca529 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:27.303699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-642c526e-a27e-42b8-a76d-6d0421dca529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:27.321161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:27.401635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:27.438304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec32a68-791d-4366-9a48-54385404e9d8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:27.439882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eec32a68-791d-4366-9a48-54385404e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:27.545155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec32a68-791d-4366-9a48-54385404e9d8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:27.611609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:27.618276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724eddd-82fc-4aef-bb95-6fe2419d5b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:27.618924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724eddd-82fc-4aef-bb95-6fe2419d5b94 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:27.619211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724eddd-82fc-4aef-bb95-6fe2419d5b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:27.619484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724eddd-82fc-4aef-bb95-6fe2419d5b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:27.621155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6724eddd-82fc-4aef-bb95-6fe2419d5b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:27.728874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724eddd-82fc-4aef-bb95-6fe2419d5b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:27.730320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724eddd-82fc-4aef-bb95-6fe2419d5b94 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:27.747933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 743/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:29.757286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b025a-72c8-4786-b2db-5ad4a3a56a64 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:29.757795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b025a-72c8-4786-b2db-5ad4a3a56a64 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:29.757795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b025a-72c8-4786-b2db-5ad4a3a56a64 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:29.759131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:29.764732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019f94-a962-4524-b86b-a110bdc61815 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:29.765143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019f94-a962-4524-b86b-a110bdc61815 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:29.765271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019f94-a962-4524-b86b-a110bdc61815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:29.831499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f6019f94-a962-4524-b86b-a110bdc61815 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:29.832715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f6019f94-a962-4524-b86b-a110bdc61815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:29.884012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:29.931935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:29.967906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b025a-72c8-4786-b2db-5ad4a3a56a64 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:29.969434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-af3b025a-72c8-4786-b2db-5ad4a3a56a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:30.070398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b025a-72c8-4786-b2db-5ad4a3a56a64 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:30.117184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:30.143514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e8e6d-15f5-4556-8c62-13c10fb17ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:30.144132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8e6d-15f5-4556-8c62-13c10fb17ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:30.144411 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8e6d-15f5-4556-8c62-13c10fb17ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:30.144680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8e6d-15f5-4556-8c62-13c10fb17ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:30.146299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ad4e8e6d-15f5-4556-8c62-13c10fb17ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:30.253262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e8e6d-15f5-4556-8c62-13c10fb17ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:30.254695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e8e6d-15f5-4556-8c62-13c10fb17ed3 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:30.272687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 726/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:39.930119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=22 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:40.270317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=66 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:49.643703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75455d03-926e-412c-acc0-795b7159a1e8 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:49.644278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75455d03-926e-412c-acc0-795b7159a1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:49.644278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75455d03-926e-412c-acc0-795b7159a1e8 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:49.645656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 745/1471] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:49.650668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff8938-c54e-48cb-9f20-c02b21c80806 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:49.651077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff8938-c54e-48cb-9f20-c02b21c80806 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:49.651242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff8938-c54e-48cb-9f20-c02b21c80806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:49.716075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f5ff8938-c54e-48cb-9f20-c02b21c80806 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:49.717269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f5ff8938-c54e-48cb-9f20-c02b21c80806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:49.813427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 727/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:49.917901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e176fb8-0d84-41c9-8688-65464feece1a None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:49.918182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e176fb8-0d84-41c9-8688-65464feece1a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:49.918454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e176fb8-0d84-41c9-8688-65464feece1a None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:49.919827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 746/1473] 10.222.0.17 () {54 vars in 966 bytes} [Sat Aug 6 14:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:49.924567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542fe1-edff-41f1-bb7c-a4cf6f6ba147 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:49.924854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542fe1-edff-41f1-bb7c-a4cf6f6ba147 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:49.925128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542fe1-edff-41f1-bb7c-a4cf6f6ba147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:49.998183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3e542fe1-edff-41f1-bb7c-a4cf6f6ba147 None None] MFA Rules not processed for user `cf386f112e1b49fdba24871681961e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:49.999404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3e542fe1-edff-41f1-bb7c-a4cf6f6ba147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:50.020523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:50.074746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 728/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Sat Aug 6 14:24:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:50.082042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e176fb8-0d84-41c9-8688-65464feece1a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:50.083612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7e176fb8-0d84-41c9-8688-65464feece1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:50.199009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e176fb8-0d84-41c9-8688-65464feece1a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:50.289703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04e68e-f8f4-4e6b-a56c-aef3a2f34da4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:50.290584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04e68e-f8f4-4e6b-a56c-aef3a2f34da4 service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:50.291055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04e68e-f8f4-4e6b-a56c-aef3a2f34da4 service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:50.291969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04e68e-f8f4-4e6b-a56c-aef3a2f34da4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:50.294446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dc04e68e-f8f4-4e6b-a56c-aef3a2f34da4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:50.348765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:50.403267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04e68e-f8f4-4e6b-a56c-aef3a2f34da4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:50.404679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04e68e-f8f4-4e6b-a56c-aef3a2f34da4 service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:50.422518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 747/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:24:52.527683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff81faf-a064-47b4-a967-f26ab92cc9a8 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:52.528077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff81faf-a064-47b4-a967-f26ab92cc9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:52.528164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff81faf-a064-47b4-a967-f26ab92cc9a8 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:52.529842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 729/1476] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:52.535425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e353bc-1b7d-4df6-957c-b1ea4882ad9c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:52.535898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e353bc-1b7d-4df6-957c-b1ea4882ad9c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:52.536002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e353bc-1b7d-4df6-957c-b1ea4882ad9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:52.606017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-48e353bc-1b7d-4df6-957c-b1ea4882ad9c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:52.607282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-48e353bc-1b7d-4df6-957c-b1ea4882ad9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:52.704125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 748/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:52.727611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff81faf-a064-47b4-a967-f26ab92cc9a8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:52.729762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2ff81faf-a064-47b4-a967-f26ab92cc9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:52.853263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff81faf-a064-47b4-a967-f26ab92cc9a8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:52.900348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:52.945281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def7fe5-bced-4843-9065-b572b10592ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:52.945904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def7fe5-bced-4843-9065-b572b10592ec admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:52.946183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def7fe5-bced-4843-9065-b572b10592ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:52.946453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def7fe5-bced-4843-9065-b572b10592ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:52.947181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def7fe5-bced-4843-9065-b572b10592ec admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:52.948918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-8def7fe5-bced-4843-9065-b572b10592ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:24:52.948918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:24:52.948918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:24:52.950045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def7fe5-bced-4843-9065-b572b10592ec admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:52.969710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 730/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Aug 6 14:24:52 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:24:54.994735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e8fbc-e1de-41ee-8da2-da0d70e11492 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:54.995342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e8fbc-e1de-41ee-8da2-da0d70e11492 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:54.995342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e8fbc-e1de-41ee-8da2-da0d70e11492 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:54.996656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 749/1479] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:55.001707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508010cb-1647-4688-b919-ba7e3fe48e50 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:55.002148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508010cb-1647-4688-b919-ba7e3fe48e50 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:55.002266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508010cb-1647-4688-b919-ba7e3fe48e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:55.066720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-508010cb-1647-4688-b919-ba7e3fe48e50 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:55.067953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-508010cb-1647-4688-b919-ba7e3fe48e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:55.164372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 731/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:55.208690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e8fbc-e1de-41ee-8da2-da0d70e11492 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:55.210305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-659e8fbc-e1de-41ee-8da2-da0d70e11492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:55.220290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:55.352934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e8fbc-e1de-41ee-8da2-da0d70e11492 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:55.431110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:55.463692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35a99f-1c8b-4623-9ca0-e3d2f09cfa25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:55.464306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35a99f-1c8b-4623-9ca0-e3d2f09cfa25 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:55.464586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35a99f-1c8b-4623-9ca0-e3d2f09cfa25 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:55.464861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35a99f-1c8b-4623-9ca0-e3d2f09cfa25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:55.466466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0b35a99f-1c8b-4623-9ca0-e3d2f09cfa25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:55.554990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35a99f-1c8b-4623-9ca0-e3d2f09cfa25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:55.556565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35a99f-1c8b-4623-9ca0-e3d2f09cfa25 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:55.574800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 750/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:24:57.747690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc8046-89f4-4710-94eb-aa9f16e120dd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:57.748253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc8046-89f4-4710-94eb-aa9f16e120dd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:57.748253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc8046-89f4-4710-94eb-aa9f16e120dd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:57.749646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 732/1482] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:24:57.754692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b99a5-e77f-4fd9-821d-ab281ef1bf64 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:57.755228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b99a5-e77f-4fd9-821d-ab281ef1bf64 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:57.755333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b99a5-e77f-4fd9-821d-ab281ef1bf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:57.819469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-756b99a5-e77f-4fd9-821d-ab281ef1bf64 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:24:57.820577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-756b99a5-e77f-4fd9-821d-ab281ef1bf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:57.874745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:57.924611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 751/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:24:57.961282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc8046-89f4-4710-94eb-aa9f16e120dd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:24:57.962853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-56cc8046-89f4-4710-94eb-aa9f16e120dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:57.985681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:58.065057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc8046-89f4-4710-94eb-aa9f16e120dd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:24:58.137522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074a43b-4cf3-4a15-97e1-729058f08431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:24:58.138196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074a43b-4cf3-4a15-97e1-729058f08431 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:24:58.138475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074a43b-4cf3-4a15-97e1-729058f08431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:24:58.138747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074a43b-4cf3-4a15-97e1-729058f08431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:24:58.140398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0074a43b-4cf3-4a15-97e1-729058f08431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:24:58.216076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:24:58.251058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074a43b-4cf3-4a15-97e1-729058f08431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:24:58.252522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074a43b-4cf3-4a15-97e1-729058f08431 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:24:58.272016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 733/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:25:00.327013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c992c-5238-4546-9995-069876cf8944 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:00.327599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c992c-5238-4546-9995-069876cf8944 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:00.327599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c992c-5238-4546-9995-069876cf8944 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:00.328947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 752/1485] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:25:00.334131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77debb-4a1d-4295-b0c8-c56a5738a177 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:00.334554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77debb-4a1d-4295-b0c8-c56a5738a177 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:00.334677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77debb-4a1d-4295-b0c8-c56a5738a177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:00.398972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4d77debb-4a1d-4295-b0c8-c56a5738a177 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:25:00.400163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4d77debb-4a1d-4295-b0c8-c56a5738a177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:00.495304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 734/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:25:00.534021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c992c-5238-4546-9995-069876cf8944 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:25:00.535580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-118c992c-5238-4546-9995-069876cf8944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:00.637987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c992c-5238-4546-9995-069876cf8944 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:25:00.707084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:25:00.708816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d002b1-66a5-4d62-bdc0-7fabf104eacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:25:00.709507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d002b1-66a5-4d62-bdc0-7fabf104eacb service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:00.709965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d002b1-66a5-4d62-bdc0-7fabf104eacb service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:00.710461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d002b1-66a5-4d62-bdc0-7fabf104eacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:00.712261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-99d002b1-66a5-4d62-bdc0-7fabf104eacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:00.818268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d002b1-66a5-4d62-bdc0-7fabf104eacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:25:00.819751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d002b1-66a5-4d62-bdc0-7fabf104eacb service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:25:00.838594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 753/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:25:02.982979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf91c2b-769b-4d8a-9d46-11ae9906878d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:02.983519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf91c2b-769b-4d8a-9d46-11ae9906878d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:02.983519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf91c2b-769b-4d8a-9d46-11ae9906878d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:02.984891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 735/1488] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:25:02.990299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569033a7-e6e6-4aae-bda0-738a072dbbe7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:02.990721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569033a7-e6e6-4aae-bda0-738a072dbbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:02.990903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569033a7-e6e6-4aae-bda0-738a072dbbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:03.057658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-569033a7-e6e6-4aae-bda0-738a072dbbe7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:25:03.058835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-569033a7-e6e6-4aae-bda0-738a072dbbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:03.109295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:25:03.154044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 754/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:25:03.195194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf91c2b-769b-4d8a-9d46-11ae9906878d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:25:03.196787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bdf91c2b-769b-4d8a-9d46-11ae9906878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:03.237097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:25:03.304029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf91c2b-769b-4d8a-9d46-11ae9906878d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:25:03.388240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7075bb8-7c70-4f7e-b125-f157a040ea07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:25:03.388863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7075bb8-7c70-4f7e-b125-f157a040ea07 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:03.389145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7075bb8-7c70-4f7e-b125-f157a040ea07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:03.389423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7075bb8-7c70-4f7e-b125-f157a040ea07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:03.391096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e7075bb8-7c70-4f7e-b125-f157a040ea07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:03.473896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:25:03.504633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7075bb8-7c70-4f7e-b125-f157a040ea07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:25:03.505985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7075bb8-7c70-4f7e-b125-f157a040ea07 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:25:03.527162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 736/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:25:05.782993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e15b1-8350-4ef6-9382-c94f77007aba None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:05.783536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e15b1-8350-4ef6-9382-c94f77007aba None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:05.783536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e15b1-8350-4ef6-9382-c94f77007aba None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:05.785379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 755/1491] 10.222.0.17 () {54 vars in 927 bytes} [Sat Aug 6 14:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:25:05.790362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3479d9-5b0a-47b9-ac0d-8e18fc794685 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:05.790804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3479d9-5b0a-47b9-ac0d-8e18fc794685 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:05.791008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3479d9-5b0a-47b9-ac0d-8e18fc794685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:05.857673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6d3479d9-5b0a-47b9-ac0d-8e18fc794685 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:25:05.858889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6d3479d9-5b0a-47b9-ac0d-8e18fc794685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:05.977331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 737/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Aug 6 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:25:06.017343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e15b1-8350-4ef6-9382-c94f77007aba None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:25:06.018917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-165e15b1-8350-4ef6-9382-c94f77007aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:06.123707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e15b1-8350-4ef6-9382-c94f77007aba None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:25:06.196615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e2c9b-b461-4a8d-a4a0-d1684f23fab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:25:06.197945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e2c9b-b461-4a8d-a4a0-d1684f23fab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:25:06.198224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e2c9b-b461-4a8d-a4a0-d1684f23fab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:25:06.198616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e2c9b-b461-4a8d-a4a0-d1684f23fab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:25:06.200515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-219e2c9b-b461-4a8d-a4a0-d1684f23fab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:25:06.209777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:25:06.313242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e2c9b-b461-4a8d-a4a0-d1684f23fab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:25:06.314670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e2c9b-b461-4a8d-a4a0-d1684f23fab5 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:25:06.333029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 756/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:25:15.975340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=82 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:25:16.330686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=50 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:23.864049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:23.864514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:23.864662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:23.938493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:26:23.939722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:24.034580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 738/1494] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:26:24.043042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920200d-747c-411d-934a-1506e7a69c2a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:24.043539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920200d-747c-411d-934a-1506e7a69c2a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:24.043645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920200d-747c-411d-934a-1506e7a69c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:24.118171 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b920200d-747c-411d-934a-1506e7a69c2a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:26:24.119327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b920200d-747c-411d-934a-1506e7a69c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:24.213972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 757/1495] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:26:24.218740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:24.219954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:24.231132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edd687-d910-4220-9d58-3e53766f57a0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:24.290180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:24.323689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:24.324292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:24.324566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:24.324836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:24.325513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:26:24.325812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:24.326811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:24.326811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:26:24.326811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:26:24.327711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:24.338551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 739/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:26:24.347038 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920200d-747c-411d-934a-1506e7a69c2a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:24.356228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920200d-747c-411d-934a-1506e7a69c2a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:24.416507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:24.450853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:24.451491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:24.451770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:24.452045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:24.452710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:24.454424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:24.454424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Aug 06 14:26:24.454424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: Aug 06 14:26:24.455656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:24.511435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 758/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:26:24.515924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:24.517730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:24.551295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3030-1301-4bfb-8c36-b6193b014620 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:24.645064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:24.645674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:24.645949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:24.646214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:24.646897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:24.647901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:24.647901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:26:24.647901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:26:24.649096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:24.690486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 740/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:26:24 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:26:24.694616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:24.696139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:24.756807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbb85-d7a9-48ba-abff-26d4511f235d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:24.849294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:24.857061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:24.857846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:24.858364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:24.858694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:24.859664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:24.860702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:24.860702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Aug 06 14:26:24.860702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: Aug 06 14:26:24.861581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:24.874402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 759/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:26:24.878555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:24.880096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:24.891373 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328ed39-ddd4-46d2-965f-656fe4c080bb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:24.916939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:24.985692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b664a4a-ae71-4661-a72c-f726c91f1b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:24.986242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664a4a-ae71-4661-a72c-f726c91f1b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:24.986520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664a4a-ae71-4661-a72c-f726c91f1b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:24.986792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664a4a-ae71-4661-a72c-f726c91f1b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:24.987511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b664a4a-ae71-4661-a72c-f726c91f1b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:24.988520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-1b664a4a-ae71-4661-a72c-f726c91f1b64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:24.988520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:26:24.988520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:26:24.989407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b664a4a-ae71-4661-a72c-f726c91f1b64 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:25.001392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 741/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:26:25.005562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:25.007125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:25.040417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a21c3-fe72-46b0-be6c-69ab4a775c41 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:25.131520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e770cf6-2e59-43b4-9d14-5c0225806f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:25.132182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e770cf6-2e59-43b4-9d14-5c0225806f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:25.132458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e770cf6-2e59-43b4-9d14-5c0225806f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:25.132755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e770cf6-2e59-43b4-9d14-5c0225806f69 admin admin] PATH_INFO: `/v3/projects/1dda4ff2ffac49d183988698acb78ce0/users/b6f12c81032b4760a7758a37cf85473f/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:25.155340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e770cf6-2e59-43b4-9d14-5c0225806f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dda4ff2ffac49d183988698acb78ce0, user_id=b6f12c81032b4760a7758a37cf85473f, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:25.156386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-3e770cf6-2e59-43b4-9d14-5c0225806f69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:25.156386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Aug 06 14:26:25.156386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: Aug 06 14:26:25.158340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e770cf6-2e59-43b4-9d14-5c0225806f69 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:25.183989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 760/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:26:25 2022] PUT /identity/v3/projects/1dda4ff2ffac49d183988698acb78ce0/users/b6f12c81032b4760a7758a37cf85473f/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:26:25.196093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d3dc-8876-41a4-a7ca-33128877dc77 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:25.196376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d3dc-8876-41a4-a7ca-33128877dc77 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:25.196648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d3dc-8876-41a4-a7ca-33128877dc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:25.251953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bcc4d3dc-8876-41a4-a7ca-33128877dc77 None None] MFA Rules not processed for user `b6f12c81032b4760a7758a37cf85473f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:26:25.253072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bcc4d3dc-8876-41a4-a7ca-33128877dc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:25.327663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 742/1502] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:26:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:26:25.341324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5686a-8e5a-4c2f-8221-bacd3617d177 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:25.341617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5686a-8e5a-4c2f-8221-bacd3617d177 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:25.341897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5686a-8e5a-4c2f-8221-bacd3617d177 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:25.343279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 761/1503] 10.222.0.17 () {54 vars in 964 bytes} [Sat Aug 6 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:26:25.348313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6c6b-eab2-4f45-9a28-d73df43292b8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:25.348603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6c6b-eab2-4f45-9a28-d73df43292b8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:25.348882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6c6b-eab2-4f45-9a28-d73df43292b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:25.379221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:25.425180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ab5e6c6b-eab2-4f45-9a28-d73df43292b8 None None] MFA Rules not processed for user `8722d97d3ddc49a082ef84060a30d1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:26:25.426388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ab5e6c6b-eab2-4f45-9a28-d73df43292b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:25.501757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 743/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Aug 6 14:26:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:26:25.510052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5686a-8e5a-4c2f-8221-bacd3617d177 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:25.511501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-68a5686a-8e5a-4c2f-8221-bacd3617d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:25.561296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:25.615353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5686a-8e5a-4c2f-8221-bacd3617d177 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:25.687210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:25.687823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:25.688102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:25.688372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:25.689976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:25.785782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:25.787193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:25.791061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:25.806757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 762/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:26:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:26:25.851284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accd017-eb04-4893-a209-3f002057234f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:25.851606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accd017-eb04-4893-a209-3f002057234f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:25.851898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accd017-eb04-4893-a209-3f002057234f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:25.853194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 744/1506] 10.222.0.17 () {52 vars in 830 bytes} [Sat Aug 6 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:26:25.856871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:25.858097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:25.955141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf049-8149-461e-ae7a-c5ba87029fc1 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:26.046472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:26.047081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:26.047361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:26.047632 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] PATH_INFO: `/v3/users/b6f12c81032b4760a7758a37cf85473f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:26.060681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6f12c81032b4760a7758a37cf85473f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:26.061699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:26.061699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Aug 06 14:26:26.061699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: Aug 06 14:26:26.062941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:26.072560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:26.129681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6f12c81032b4760a7758a37cf85473f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:26.130214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6f12c81032b4760a7758a37cf85473f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:26.130670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6f12c81032b4760a7758a37cf85473f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:26.153326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6f12c81032b4760a7758a37cf85473f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:26.167196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6f12c81032b4760a7758a37cf85473f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:26.182136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 763/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:26:25 2022] DELETE /identity/v3/users/b6f12c81032b4760a7758a37cf85473f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:26:26.186528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:26.187028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:26.187169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:26.242666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:26:26.244519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:26.318828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:26.331256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 745/1508] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:26:26.340767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:26.341980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:26.440215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fb5c2-128f-4c11-b80c-80e12047a174 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:26.494914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:26.515305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409fd8d-0e4b-483c-8f1a-18c0b0980f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:26.515854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409fd8d-0e4b-483c-8f1a-18c0b0980f9c service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:26.516140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409fd8d-0e4b-483c-8f1a-18c0b0980f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:26.516393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409fd8d-0e4b-483c-8f1a-18c0b0980f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:26.518140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2409fd8d-0e4b-483c-8f1a-18c0b0980f9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:26.623570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409fd8d-0e4b-483c-8f1a-18c0b0980f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:26.624912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409fd8d-0e4b-483c-8f1a-18c0b0980f9c service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:26.642763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 764/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:26:27.075269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:26:27.076509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:26:27.089136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35092f9-b166-4fca-99db-e9d55cd13254 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:26:27.192739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:26:27.193399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:26:27.193565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:26:27.193836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] PATH_INFO: `/v3/projects/1dda4ff2ffac49d183988698acb78ce0` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:26:27.204259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dda4ff2ffac49d183988698acb78ce0)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:26:27.205281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:26:27.205281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:26:27.205281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:26:27.206473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:26:27.262983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dda4ff2ffac49d183988698acb78ce0', 'request_id': 'req-2e270735-a323-434d-adc1-799cd519ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:27.275832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dda4ff2ffac49d183988698acb78ce0', 'request_id': 'req-2e270735-a323-434d-adc1-799cd519ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:27.284434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dda4ff2ffac49d183988698acb78ce0', 'request_id': 'req-2e270735-a323-434d-adc1-799cd519ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:27.307167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] The token cache is being invalidate because project 1dda4ff2ffac49d183988698acb78ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:26:27.307455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:27.307969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2e270735-a323-434d-adc1-799cd519ce3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:26:27.309313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 746/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:26:27 2022] DELETE /identity/v3/projects/1dda4ff2ffac49d183988698acb78ce0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:26:36.640523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=62 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:26:37.304756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:27:52.063121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cfe7d-9e0f-411f-808f-8f51bb98f3b6 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:52.063724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cfe7d-9e0f-411f-808f-8f51bb98f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:52.063724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cfe7d-9e0f-411f-808f-8f51bb98f3b6 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:52.065028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:27:52.069933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988a83c-f3aa-4027-9660-0737b41423da None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:52.070395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988a83c-f3aa-4027-9660-0737b41423da None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:52.070499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988a83c-f3aa-4027-9660-0737b41423da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:52.152691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2988a83c-f3aa-4027-9660-0737b41423da None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:27:52.153919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2988a83c-f3aa-4027-9660-0737b41423da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:27:52.254395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:27:52.405784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cfe7d-9e0f-411f-808f-8f51bb98f3b6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:27:52.407945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-896cfe7d-9e0f-411f-808f-8f51bb98f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:27:52.525760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cfe7d-9e0f-411f-808f-8f51bb98f3b6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:27:52.531594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a97f4-4f2d-4837-a969-c8c3e356a24f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:27:52.532226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a97f4-4f2d-4837-a969-c8c3e356a24f service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:52.532539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a97f4-4f2d-4837-a969-c8c3e356a24f service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:52.532818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a97f4-4f2d-4837-a969-c8c3e356a24f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:52.541546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a97f4-4f2d-4837-a969-c8c3e356a24f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:27:52.543201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a97f4-4f2d-4837-a969-c8c3e356a24f service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:27:52.552892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 766/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:27:53.393160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7b35d-4771-4c9d-861d-54e01c190c16 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:53.393717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7b35d-4771-4c9d-861d-54e01c190c16 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:53.393717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7b35d-4771-4c9d-861d-54e01c190c16 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:53.395113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:27:53.400050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f8e44-fb99-4268-963d-b4d3a74eb20d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:53.401026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f8e44-fb99-4268-963d-b4d3a74eb20d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:53.401134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f8e44-fb99-4268-963d-b4d3a74eb20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:53.420704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5b8f8e44-fb99-4268-963d-b4d3a74eb20d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:27:53.421894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5b8f8e44-fb99-4268-963d-b4d3a74eb20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:27:53.429926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:27:53.580764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7b35d-4771-4c9d-861d-54e01c190c16 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:27:53.595511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7b35d-4771-4c9d-861d-54e01c190c16 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:27:53.600802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a17c1b-ba50-4c03-856f-aea08c6c0671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:27:53.601407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a17c1b-ba50-4c03-856f-aea08c6c0671 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:53.601682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a17c1b-ba50-4c03-856f-aea08c6c0671 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:53.601952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a17c1b-ba50-4c03-856f-aea08c6c0671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:53.610635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a17c1b-ba50-4c03-856f-aea08c6c0671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:27:53.612249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a17c1b-ba50-4c03-856f-aea08c6c0671 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:27:53.621457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 749/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:27:54.036377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6824d8e-1a89-46b1-bef6-164cedc264a0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:54.036596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6824d8e-1a89-46b1-bef6-164cedc264a0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:54.036882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6824d8e-1a89-46b1-bef6-164cedc264a0 None None] PATH_INFO: `/v3` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:54.038210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 768/1517] 10.222.0.36 () {54 vars in 925 bytes} [Sat Aug 6 14:27:54 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:27:54.045638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cd90b-c545-4b12-bd81-bc4c79541985 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:27:54.045924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cd90b-c545-4b12-bd81-bc4c79541985 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:27:54.046197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cd90b-c545-4b12-bd81-bc4c79541985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:27:54.104894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c14cd90b-c545-4b12-bd81-bc4c79541985 None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:27:54.106132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c14cd90b-c545-4b12-bd81-bc4c79541985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:27:54.123050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:27:54.176581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 750/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Sat Aug 6 14:27:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:03.121117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1e75-365b-4621-a4ac-654640c46e27 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:03.121642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1e75-365b-4621-a4ac-654640c46e27 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:03.121642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1e75-365b-4621-a4ac-654640c46e27 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:03.122914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 769/1519] 10.222.0.17 () {54 vars in 961 bytes} [Sat Aug 6 14:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:28:03.127923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05145866-819c-443a-9ca2-866143626cc8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:03.128340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05145866-819c-443a-9ca2-866143626cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:03.128510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05145866-819c-443a-9ca2-866143626cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:03.147393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-05145866-819c-443a-9ca2-866143626cc8 None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:28:03.148610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-05145866-819c-443a-9ca2-866143626cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:03.157550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 751/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Aug 6 14:28:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:03.164820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1e75-365b-4621-a4ac-654640c46e27 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:28:03.178379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1e75-365b-4621-a4ac-654640c46e27 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:28:03.183618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c63433-1fbf-46c9-b6e9-112fba70953a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:28:03.184205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c63433-1fbf-46c9-b6e9-112fba70953a service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:03.184483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c63433-1fbf-46c9-b6e9-112fba70953a service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:03.184755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c63433-1fbf-46c9-b6e9-112fba70953a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:03.192730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c63433-1fbf-46c9-b6e9-112fba70953a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:28:03.194282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c63433-1fbf-46c9-b6e9-112fba70953a service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:28:03.203620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 770/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:28:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:28:04.862760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a03cb5-f7df-430c-979c-5e84e1fb9938 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:04.863304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a03cb5-f7df-430c-979c-5e84e1fb9938 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:04.863414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a03cb5-f7df-430c-979c-5e84e1fb9938 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:04.864782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:28:04.869972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bc344-127b-4173-a270-5894db6cebbe None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:04.870438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bc344-127b-4173-a270-5894db6cebbe None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:04.870568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bc344-127b-4173-a270-5894db6cebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:04.890039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-621bc344-127b-4173-a270-5894db6cebbe None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:28:04.891271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-621bc344-127b-4173-a270-5894db6cebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:04.899018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:05.051424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a03cb5-f7df-430c-979c-5e84e1fb9938 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:28:05.066279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a03cb5-f7df-430c-979c-5e84e1fb9938 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:28:05.071657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7d773-2202-4e89-a084-b4b690eb2115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:28:05.072285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d773-2202-4e89-a084-b4b690eb2115 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:05.072562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d773-2202-4e89-a084-b4b690eb2115 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:05.072860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d773-2202-4e89-a084-b4b690eb2115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:05.081083 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7d773-2202-4e89-a084-b4b690eb2115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:28:05.082660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7d773-2202-4e89-a084-b4b690eb2115 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:28:05.092058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 753/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:28:05.914712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1580e-e2b9-4094-a566-8d834ad0b5fe None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:05.915298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1580e-e2b9-4094-a566-8d834ad0b5fe None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:05.915298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1580e-e2b9-4094-a566-8d834ad0b5fe None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:05.916640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 14:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:28:05.921448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7287a62-53bd-4e75-9fcd-9bcb3b2e6693 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:05.921872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7287a62-53bd-4e75-9fcd-9bcb3b2e6693 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:05.921998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7287a62-53bd-4e75-9fcd-9bcb3b2e6693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:05.941108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d7287a62-53bd-4e75-9fcd-9bcb3b2e6693 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:28:05.942292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d7287a62-53bd-4e75-9fcd-9bcb3b2e6693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:05.950686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:06.104612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1580e-e2b9-4094-a566-8d834ad0b5fe None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:28:06.119757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1580e-e2b9-4094-a566-8d834ad0b5fe None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:28:06.125317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bab9f-960a-49b9-9b01-a900712a57d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:28:06.125954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bab9f-960a-49b9-9b01-a900712a57d2 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:06.126238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bab9f-960a-49b9-9b01-a900712a57d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:06.126514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bab9f-960a-49b9-9b01-a900712a57d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:06.135295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bab9f-960a-49b9-9b01-a900712a57d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:28:06.136915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bab9f-960a-49b9-9b01-a900712a57d2 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:28:06.146172 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 773/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:28:09.874155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ff227-42e5-4aea-97e0-ca61db6914fd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:09.874669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ff227-42e5-4aea-97e0-ca61db6914fd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:09.874669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ff227-42e5-4aea-97e0-ca61db6914fd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:09.876031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:28:09.881242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e1f99-5af8-43bb-bfbd-3ff5494f506c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:09.881682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e1f99-5af8-43bb-bfbd-3ff5494f506c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:09.881790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e1f99-5af8-43bb-bfbd-3ff5494f506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:09.901434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ec5e1f99-5af8-43bb-bfbd-3ff5494f506c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:28:09.902619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ec5e1f99-5af8-43bb-bfbd-3ff5494f506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:09.910420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:09.984111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ff227-42e5-4aea-97e0-ca61db6914fd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:28:09.985873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ea7ff227-42e5-4aea-97e0-ca61db6914fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:10.065603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ff227-42e5-4aea-97e0-ca61db6914fd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:28:10.069114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86af3-825b-4c5f-ad99-8d8430a3daa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:28:10.069727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86af3-825b-4c5f-ad99-8d8430a3daa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:10.070004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86af3-825b-4c5f-ad99-8d8430a3daa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:10.070275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86af3-825b-4c5f-ad99-8d8430a3daa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:10.079251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad86af3-825b-4c5f-ad99-8d8430a3daa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:28:10.080753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad86af3-825b-4c5f-ad99-8d8430a3daa8 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:28:10.090578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 756/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:28:12.076955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c07c87-6fe1-45d0-b632-ac9b78693c2c None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:12.077493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c07c87-6fe1-45d0-b632-ac9b78693c2c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:12.077493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c07c87-6fe1-45d0-b632-ac9b78693c2c None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:12.078983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:28:12.084160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc818-5252-461b-9e55-74dba071b288 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:12.084564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc818-5252-461b-9e55-74dba071b288 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:12.084714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc818-5252-461b-9e55-74dba071b288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:12.103805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-69cdc818-5252-461b-9e55-74dba071b288 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:28:12.105002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-69cdc818-5252-461b-9e55-74dba071b288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:12.112852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:12.148705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c07c87-6fe1-45d0-b632-ac9b78693c2c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:28:12.159364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c07c87-6fe1-45d0-b632-ac9b78693c2c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:28:12.162894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58ccaa-65f8-496c-bfb1-79b74c51aecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:28:12.163499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58ccaa-65f8-496c-bfb1-79b74c51aecd service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:12.163782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58ccaa-65f8-496c-bfb1-79b74c51aecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:12.164086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58ccaa-65f8-496c-bfb1-79b74c51aecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:12.172788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58ccaa-65f8-496c-bfb1-79b74c51aecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:28:12.174194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58ccaa-65f8-496c-bfb1-79b74c51aecd service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:28:12.184130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 776/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:28:15.677209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4474-ad72-4bab-ba17-f9bce5f2a4d9 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:15.677702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4474-ad72-4bab-ba17-f9bce5f2a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:15.677797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4474-ad72-4bab-ba17-f9bce5f2a4d9 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:15.679409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:28:15.685287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac26463-f244-4765-92ed-76ef10230895 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:15.685687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac26463-f244-4765-92ed-76ef10230895 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:15.685872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac26463-f244-4765-92ed-76ef10230895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:15.704740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4ac26463-f244-4765-92ed-76ef10230895 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:28:15.705928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4ac26463-f244-4765-92ed-76ef10230895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:15.713703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:16.018024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4474-ad72-4bab-ba17-f9bce5f2a4d9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:28:16.019778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8e0b4474-ad72-4bab-ba17-f9bce5f2a4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:16.086800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:28:16.100407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4474-ad72-4bab-ba17-f9bce5f2a4d9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:28:16.104550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc954b98-d6c2-45e6-9065-738c5c994caf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:28:16.105171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc954b98-d6c2-45e6-9065-738c5c994caf service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:16.105436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc954b98-d6c2-45e6-9065-738c5c994caf service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:16.105705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc954b98-d6c2-45e6-9065-738c5c994caf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:16.114515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc954b98-d6c2-45e6-9065-738c5c994caf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:28:16.116027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc954b98-d6c2-45e6-9065-738c5c994caf service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:28:16.125548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 759/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:28:18.137758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5ef75-92eb-4cfe-abd2-787629cca9d4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:18.138324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5ef75-92eb-4cfe-abd2-787629cca9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:18.138324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5ef75-92eb-4cfe-abd2-787629cca9d4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:18.139718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 14:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:28:18.144750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b983c6b-00d4-4a38-a218-b2e8dc8d167d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:18.145158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b983c6b-00d4-4a38-a218-b2e8dc8d167d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:18.145302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b983c6b-00d4-4a38-a218-b2e8dc8d167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:18.164820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5b983c6b-00d4-4a38-a218-b2e8dc8d167d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:28:18.166687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5b983c6b-00d4-4a38-a218-b2e8dc8d167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:18.174446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 14:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:28:18.475772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5ef75-92eb-4cfe-abd2-787629cca9d4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:28:18.478181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eaa5ef75-92eb-4cfe-abd2-787629cca9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:28:18.486306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:28:18.492527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5ef75-92eb-4cfe-abd2-787629cca9d4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:28:18.495964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837738c-9749-44b8-bc5d-36c434accc04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:28:18.496565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837738c-9749-44b8-bc5d-36c434accc04 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:28:18.496871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837738c-9749-44b8-bc5d-36c434accc04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:28:18.497143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837738c-9749-44b8-bc5d-36c434accc04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:28:18.505796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837738c-9749-44b8-bc5d-36c434accc04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:28:18.507257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837738c-9749-44b8-bc5d-36c434accc04 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:28:18.516841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 779/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:28:28.159811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=11 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:28:28.512854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:40.969922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9c885-6b34-4afb-b9af-b571bb768a06 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:40.970579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9c885-6b34-4afb-b9af-b571bb768a06 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:40.970579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9c885-6b34-4afb-b9af-b571bb768a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:40.996638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-40c9c885-6b34-4afb-b9af-b571bb768a06 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:40.997869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-40c9c885-6b34-4afb-b9af-b571bb768a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.008539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 761/1540] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.018724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3b7a-ed49-4d9c-b2a3-f50f86afad59 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.019348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3b7a-ed49-4d9c-b2a3-f50f86afad59 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.019348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3b7a-ed49-4d9c-b2a3-f50f86afad59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.052851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-070c3b7a-ed49-4d9c-b2a3-f50f86afad59 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.054493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-070c3b7a-ed49-4d9c-b2a3-f50f86afad59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.066570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 780/1541] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.072686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9c885-6b34-4afb-b9af-b571bb768a06 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.090556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9c885-6b34-4afb-b9af-b571bb768a06 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.095673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.096337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.096615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.096891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.097636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:41.097944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.099701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.122685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 762/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:41.131973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3b7a-ed49-4d9c-b2a3-f50f86afad59 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.142588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3b7a-ed49-4d9c-b2a3-f50f86afad59 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.151521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ff53b-9319-4871-9428-c57ba96ff092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.152169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ff53b-9319-4871-9428-c57ba96ff092 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.152447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ff53b-9319-4871-9428-c57ba96ff092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.152720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ff53b-9319-4871-9428-c57ba96ff092 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.153392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ff53b-9319-4871-9428-c57ba96ff092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.155336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ff53b-9319-4871-9428-c57ba96ff092 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.213565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 781/1543] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/projects => generated 391 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:41.218302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.232907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a4f73-4afa-4c17-a5fa-276987cde22c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.296598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c801ab-7a67-4c19-b904-4eb04d0f17b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.297232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c801ab-7a67-4c19-b904-4eb04d0f17b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.297507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c801ab-7a67-4c19-b904-4eb04d0f17b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.297779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c801ab-7a67-4c19-b904-4eb04d0f17b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.298442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c801ab-7a67-4c19-b904-4eb04d0f17b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.300384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c801ab-7a67-4c19-b904-4eb04d0f17b8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.309460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835669c-5024-43f6-ae52-386059a3ffeb None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.309764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835669c-5024-43f6-ae52-386059a3ffeb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.310059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835669c-5024-43f6-ae52-386059a3ffeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.340508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5835669c-5024-43f6-ae52-386059a3ffeb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.341722 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5835669c-5024-43f6-ae52-386059a3ffeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.350726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 782/1544] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:29:41.352057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1d317-3e49-4318-ab95-48ea1aab4348 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.352340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1d317-3e49-4318-ab95-48ea1aab4348 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.352657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1d317-3e49-4318-ab95-48ea1aab4348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.361476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 763/1545] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:41.363702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353e1b-038a-4344-8e88-9fd0c8304631 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.364026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353e1b-038a-4344-8e88-9fd0c8304631 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.364244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353e1b-038a-4344-8e88-9fd0c8304631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.391072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a1d1d317-3e49-4318-ab95-48ea1aab4348 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.392695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a1d1d317-3e49-4318-ab95-48ea1aab4348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.397936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2e353e1b-038a-4344-8e88-9fd0c8304631 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.399234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2e353e1b-038a-4344-8e88-9fd0c8304631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.413375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 764/1546] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:29:41.414216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e353e1b-038a-4344-8e88-9fd0c8304631 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.415621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 783/1547] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.420664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1d317-3e49-4318-ab95-48ea1aab4348 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.426816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e353e1b-038a-4344-8e88-9fd0c8304631 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.438590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be99e3a-086a-4d59-98e3-c587f6f31c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.438590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99e3a-086a-4d59-98e3-c587f6f31c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.438590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99e3a-086a-4d59-98e3-c587f6f31c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.438590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99e3a-086a-4d59-98e3-c587f6f31c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.439096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be99e3a-086a-4d59-98e3-c587f6f31c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.439096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be99e3a-086a-4d59-98e3-c587f6f31c39 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.439760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1d317-3e49-4318-ab95-48ea1aab4348 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.452109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.453116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.453631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.454126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.455154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:41.455667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.456907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:29:41.456907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Aug 06 14:29:41.456907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: Aug 06 14:29:41.458177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.462890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 765/1548] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:41.462890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb21643-727b-4e35-afaa-c08affd7c65a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.462890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb21643-727b-4e35-afaa-c08affd7c65a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.462890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb21643-727b-4e35-afaa-c08affd7c65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.476991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 784/1549] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:29:41.482958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.496873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326358bd-b076-4838-bc13-dadc67c43f80 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.502712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45d642-86e9-4c29-ac58-e3b30c615014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.503640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45d642-86e9-4c29-ac58-e3b30c615014 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.504153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45d642-86e9-4c29-ac58-e3b30c615014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.504780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45d642-86e9-4c29-ac58-e3b30c615014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.506317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45d642-86e9-4c29-ac58-e3b30c615014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.508396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45d642-86e9-4c29-ac58-e3b30c615014 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.512108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ffb21643-727b-4e35-afaa-c08affd7c65a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.513542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ffb21643-727b-4e35-afaa-c08affd7c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.522687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 785/1550] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:41.524273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152da64e-435a-470e-9209-648dac37f35b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.524507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 766/1551] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.524712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152da64e-435a-470e-9209-648dac37f35b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.525129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152da64e-435a-470e-9209-648dac37f35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.525850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7ec2-f0f3-4a31-9e1b-6357036bf89b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.526376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7ec2-f0f3-4a31-9e1b-6357036bf89b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.526656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7ec2-f0f3-4a31-9e1b-6357036bf89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.545496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-152da64e-435a-470e-9209-648dac37f35b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.546043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-90aa7ec2-f0f3-4a31-9e1b-6357036bf89b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.546710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-152da64e-435a-470e-9209-648dac37f35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.547227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-90aa7ec2-f0f3-4a31-9e1b-6357036bf89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.555400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 767/1552] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.555531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 786/1553] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.556438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7ec2-f0f3-4a31-9e1b-6357036bf89b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.560148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789032-e21d-4eaf-a04a-479a8a30d55c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.560717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789032-e21d-4eaf-a04a-479a8a30d55c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.561252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789032-e21d-4eaf-a04a-479a8a30d55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.566126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7ec2-f0f3-4a31-9e1b-6357036bf89b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.570677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083b8ba-4af0-4401-9e7b-6f53550795e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.571379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083b8ba-4af0-4401-9e7b-6f53550795e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.572163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083b8ba-4af0-4401-9e7b-6f53550795e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.572434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083b8ba-4af0-4401-9e7b-6f53550795e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.573083 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083b8ba-4af0-4401-9e7b-6f53550795e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.574097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-3083b8ba-4af0-4401-9e7b-6f53550795e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:29:41.574097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:29:41.574097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:29:41.575299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083b8ba-4af0-4401-9e7b-6f53550795e4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.581893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7d789032-e21d-4eaf-a04a-479a8a30d55c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.583113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7d789032-e21d-4eaf-a04a-479a8a30d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.591150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 787/1554] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.592606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bc0b8-3afa-47ca-98fc-a7b501b95713 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.592949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bc0b8-3afa-47ca-98fc-a7b501b95713 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.593185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bc0b8-3afa-47ca-98fc-a7b501b95713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.605758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 768/1555] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:41.607120 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a24cd-a395-440b-84cd-62e208666b0a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.607569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a24cd-a395-440b-84cd-62e208666b0a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.608018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a24cd-a395-440b-84cd-62e208666b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.619703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7b9bc0b8-3afa-47ca-98fc-a7b501b95713 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.620918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7b9bc0b8-3afa-47ca-98fc-a7b501b95713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.629366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-879a24cd-a395-440b-84cd-62e208666b0a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.630542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-879a24cd-a395-440b-84cd-62e208666b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.684649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 788/1556] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.685553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bc0b8-3afa-47ca-98fc-a7b501b95713 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.694857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bc0b8-3afa-47ca-98fc-a7b501b95713 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.696064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 769/1557] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.696940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a24cd-a395-440b-84cd-62e208666b0a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.703491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa0109-5327-4e73-8a58-9d420029a548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.704041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0109-5327-4e73-8a58-9d420029a548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.704324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0109-5327-4e73-8a58-9d420029a548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.704606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0109-5327-4e73-8a58-9d420029a548 admin admin] PATH_INFO: `/v3/projects/d19dc46cdcf549b0a6383c8b99d6f15f/users/4dadf737392d4153ae50de020534f070/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.708554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a24cd-a395-440b-84cd-62e208666b0a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.713188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7ce88-f1c9-4e02-8e1f-35c834920cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.713797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ce88-f1c9-4e02-8e1f-35c834920cca admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.714078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ce88-f1c9-4e02-8e1f-35c834920cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.714350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ce88-f1c9-4e02-8e1f-35c834920cca admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.715093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7ce88-f1c9-4e02-8e1f-35c834920cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:41.715414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7ce88-f1c9-4e02-8e1f-35c834920cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.717188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7ce88-f1c9-4e02-8e1f-35c834920cca admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.719205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa0109-5327-4e73-8a58-9d420029a548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19dc46cdcf549b0a6383c8b99d6f15f, user_id=4dadf737392d4153ae50de020534f070, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.721835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa0109-5327-4e73-8a58-9d420029a548 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.729370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 770/1558] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:41.730569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1e6e-9666-4d4c-a1a8-46267cc41b25 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.730857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1e6e-9666-4d4c-a1a8-46267cc41b25 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.731183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1e6e-9666-4d4c-a1a8-46267cc41b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.739416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:41.743977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 789/1559] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:41 2022] PUT /identity/v3/projects/d19dc46cdcf549b0a6383c8b99d6f15f/users/4dadf737392d4153ae50de020534f070/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:41.745330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d801-4e3d-4fa5-8811-f22cbca2326f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.745617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d801-4e3d-4fa5-8811-f22cbca2326f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.745902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d801-4e3d-4fa5-8811-f22cbca2326f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.752826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-23ed1e6e-9666-4d4c-a1a8-46267cc41b25 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.754103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-23ed1e6e-9666-4d4c-a1a8-46267cc41b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.765131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8f26d801-4e3d-4fa5-8811-f22cbca2326f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.766533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8f26d801-4e3d-4fa5-8811-f22cbca2326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.811321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:41.812925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 771/1560] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.814318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33213226-0ec1-4e17-88a2-59eda63207c5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.814602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33213226-0ec1-4e17-88a2-59eda63207c5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.814924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33213226-0ec1-4e17-88a2-59eda63207c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.827940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 790/1561] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.828743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d801-4e3d-4fa5-8811-f22cbca2326f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.834433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-33213226-0ec1-4e17-88a2-59eda63207c5 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.835656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-33213226-0ec1-4e17-88a2-59eda63207c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.838573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d801-4e3d-4fa5-8811-f22cbca2326f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.843094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d43f30-46ec-4b1c-b481-c67e3cad2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.843833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 772/1562] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.843956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d43f30-46ec-4b1c-b481-c67e3cad2c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.843956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d43f30-46ec-4b1c-b481-c67e3cad2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.844218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d43f30-46ec-4b1c-b481-c67e3cad2c3f admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.844917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78905-7f4d-48c1-8a2d-3fd268d0d721 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.845019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d43f30-46ec-4b1c-b481-c67e3cad2c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.845255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78905-7f4d-48c1-8a2d-3fd268d0d721 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.845535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78905-7f4d-48c1-8a2d-3fd268d0d721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.845929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-c3d43f30-46ec-4b1c-b481-c67e3cad2c3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:29:41.845929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Aug 06 14:29:41.845929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: Aug 06 14:29:41.847376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d43f30-46ec-4b1c-b481-c67e3cad2c3f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.869405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-84a78905-7f4d-48c1-8a2d-3fd268d0d721 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.870427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 791/1563] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:41.870617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-84a78905-7f4d-48c1-8a2d-3fd268d0d721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.874606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bad8b-4366-4e82-8f0a-c424a407fa1a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.874953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bad8b-4366-4e82-8f0a-c424a407fa1a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.875235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bad8b-4366-4e82-8f0a-c424a407fa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.879114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 773/1564] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.879974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78905-7f4d-48c1-8a2d-3fd268d0d721 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.889690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78905-7f4d-48c1-8a2d-3fd268d0d721 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.894192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f14bad8b-4366-4e82-8f0a-c424a407fa1a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.894334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:41.894915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.895197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.895417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f14bad8b-4366-4e82-8f0a-c424a407fa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.895533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.896111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:41.897999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:41.903926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 792/1565] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.905159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd5f91-6e79-4c40-aec1-0e144d4239ba None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:41.905446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd5f91-6e79-4c40-aec1-0e144d4239ba None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:41.905721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd5f91-6e79-4c40-aec1-0e144d4239ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:41.925306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-63dd5f91-6e79-4c40-aec1-0e144d4239ba None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:41.925877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 774/1566] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:41.926763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.930020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-63dd5f91-6e79-4c40-aec1-0e144d4239ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:41.989694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 793/1567] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:41.990718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd5f91-6e79-4c40-aec1-0e144d4239ba None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:41.991505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc9d2-7912-4772-88a0-d3672d639532 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:41.999752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.000397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.000682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.000958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.001300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd5f91-6e79-4c40-aec1-0e144d4239ba None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.001660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:42.001964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.003600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.005920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.006537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.006813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.007114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.007801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:42.008103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.009697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.015564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 775/1568] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:42.016436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.026389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8192558-d4df-45f3-adbf-841cddefb35f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.028167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 794/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:42.028975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.036817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.037424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.037705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.037978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.038462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d75c-d709-425e-bc10-842af0f9d5ab admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.038665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:42.039002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.041263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.043630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.044231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.044493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.044794 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.045495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.047084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.053926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 776/1570] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:42.054731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.061731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 795/1571] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.061731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.065500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7814-db32-4ba5-ae58-d61a03c9fdc5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.070358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.071055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.071178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb34299-0c9f-479d-80c7-55ac43ce781c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.071318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.071576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.072266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:42.072573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.074170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.075675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.076380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.076666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.076944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.077644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:42.077950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.079548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.085908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 777/1572] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:42.086598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.096600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e745bf0-9ea0-4988-b764-5d714a89b88f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.104333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 796/1573] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:42.104333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.104581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.104581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.104581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.104581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.104928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.105530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.126901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 778/1574] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.127676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.137109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f686bd8-7423-4c94-8e36-bb1a05fd3f0d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.142068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.142765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.143084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.143358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.144489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.147152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.159574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a3b70-2fe5-4b3c-98cb-697afebf664c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.165748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.166241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.166528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.166799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.169347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:42.176259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.177623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.178036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 779/1575] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.178977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.190774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 797/1576] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:42.191586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.201714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a09466-fe24-41f7-a237-b259575d0e22 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.240242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:42.244362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd8085-1786-4713-8984-a40ee7dcc812 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.251148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.251842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.252126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.252399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.253707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.255860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.272407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.273129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.273590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.274042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.275359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.277659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.289409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 780/1577] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/projects => generated 401 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.291188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.308438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 798/1578] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.309505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.365052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005d098-bed3-4728-8b8b-458f9f9c56d9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.376615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e647b-9688-4fce-9e7e-6f5fe46d310c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.381322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.381953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.382237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.382522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.383366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.385339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.417199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 799/1579] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.418069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.443672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.444142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.444450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.445079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.445413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.447682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.463016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 781/1580] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.463016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.476041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8030-cae5-4c36-9187-80c660120b1b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.495641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c4c0a-a781-436a-9b2f-b0bdbd0bdaf6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.500339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.500957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.501237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.501505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.502234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.504273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.507976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:42.537297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 800/1581] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.538255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.551785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.552417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.552833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.553103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.553799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.555728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.566207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 782/1582] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.566965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.575269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:42.579877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbefd7-0bae-43d5-8e3a-465bdb1535d7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.615315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cad10c-1d9a-432f-83a7-f342a69a29c7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.620358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.621036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.621322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.621599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.622276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.624272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.650000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.651419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.651802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.652217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.652996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.655708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.658671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 801/1583] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.662770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.676548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd03b8-0a8d-4762-a92a-e9006bbf3f6b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.681698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.682325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.682606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.682908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.683396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 783/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/users => generated 301 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.684303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.686615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.688589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.695323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee94595-55c4-4f8f-b803-0b53c0f89cfa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.704292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.708339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.708339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.708339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.708339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.708908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.725101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 802/1585] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/users => generated 289 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.726193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.739593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 784/1586] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/users => generated 286 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.740566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.741618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a36c91-7338-4213-8ab6-dc394c3fe821 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.747200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.748503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.748503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.748711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] PATH_INFO: `/v3/projects/4ec92e2a9a9e47519d4003039ea1a0bc/users/a6275ec0936a4471a49b9a6aa1eef931/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.757039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57406146-781b-404a-b1c7-de5002773c4c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.766097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec92e2a9a9e47519d4003039ea1a0bc, user_id=a6275ec0936a4471a49b9a6aa1eef931, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.766384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.767300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.767678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.768084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.768859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.768991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.771806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.790422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 803/1587] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:42 2022] PUT /identity/v3/projects/4ec92e2a9a9e47519d4003039ea1a0bc/users/a6275ec0936a4471a49b9a6aa1eef931/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:42.791649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.810714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 785/1588] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/users => generated 289 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:42.811754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.824727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba9851-8cf0-47bf-9dce-d50c756d2826 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.883286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909874-fadc-437e-bdfd-24d3a2a52e34 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.888630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.889257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.889537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.889811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.890566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.891890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.892188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.892640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.892984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.893420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.894057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.895673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.903249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 804/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.904024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.907009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 786/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.907699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.917727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4ad4a-607e-4a93-82f1-30dc9f72cf14 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.918075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8adb5-a0a8-48ce-b119-d7557fb050a3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.922478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.922742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1716b37-0140-434a-93c6-a012d72d6940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.923290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.923419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716b37-0140-434a-93c6-a012d72d6940 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.923635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.923752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716b37-0140-434a-93c6-a012d72d6940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.923895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716b37-0140-434a-93c6-a012d72d6940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.924016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.924571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1716b37-0140-434a-93c6-a012d72d6940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.924749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.926162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1716b37-0140-434a-93c6-a012d72d6940 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.926515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.937990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 805/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.941094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 787/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.941750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.942609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafe238-d466-4ca9-b210-cce61691b6ed None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.943008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafe238-d466-4ca9-b210-cce61691b6ed None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.943274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafe238-d466-4ca9-b210-cce61691b6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.952619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26a2ad-4316-43da-8d7b-01aae4dab3c3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.952938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:42.958240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.958972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.959324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.959670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.960435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.962127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:42.973361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 806/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:42.974181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:42.977315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9cafe238-d466-4ca9-b210-cce61691b6ed None None] MFA Rules not processed for user `a6275ec0936a4471a49b9a6aa1eef931`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:42.978903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9cafe238-d466-4ca9-b210-cce61691b6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:42.984144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd55d31-87ea-40a7-8258-fa3c17cd2507 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:42.989291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:42.989931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:42.990258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:42.990570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:42.991223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:42.992930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.006692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 807/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:43.006692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.020619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598397b1-0577-4019-b569-7433910c6651 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.027221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.028353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.028911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.028911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] PATH_INFO: `/v3/projects/2f20dc7bdc3a4448a56a76f1efbec596/users/be34f77da4d8407e9a4333bc71f64595/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.046883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f20dc7bdc3a4448a56a76f1efbec596, user_id=be34f77da4d8407e9a4333bc71f64595, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.049668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.064686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 788/1595] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:43.069430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 808/1596] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:43 2022] PUT /identity/v3/projects/2f20dc7bdc3a4448a56a76f1efbec596/users/be34f77da4d8407e9a4333bc71f64595/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:43.069430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.070511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafe238-d466-4ca9-b210-cce61691b6ed None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.088261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:43.091213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafe238-d466-4ca9-b210-cce61691b6ed None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.170785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554779a6-01ef-4805-81aa-9f3bcaa5d53f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.179673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.180470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.181207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.181443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.181642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.181754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.181933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.182194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.182383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.183067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.183968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.185021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.196992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 809/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:43.198201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.200462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 789/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:43.201532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.210264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cd9a6-4682-4bb4-93dc-cf6104b6b204 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.213326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bd748-17db-4605-89c8-398561576ef3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.216215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.217262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.217814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.218310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.219230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.221084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b7eef-aab3-4d2b-9f66-78fe01626d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.221689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.221802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b7eef-aab3-4d2b-9f66-78fe01626d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.222034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b7eef-aab3-4d2b-9f66-78fe01626d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.222368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b7eef-aab3-4d2b-9f66-78fe01626d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.223145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b7eef-aab3-4d2b-9f66-78fe01626d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.225296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b7eef-aab3-4d2b-9f66-78fe01626d77 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.234414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 810/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:43.236420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.238471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 790/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:43.240170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59689c52-64ad-4f0b-8ed7-5ae4bcf0d3ac None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.240461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59689c52-64ad-4f0b-8ed7-5ae4bcf0d3ac None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.240933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59689c52-64ad-4f0b-8ed7-5ae4bcf0d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.250590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66454f6-0105-4243-8ee2-7073fbf5306a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.257497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f7c55-3251-426a-89ed-37d67536e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.258379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7c55-3251-426a-89ed-37d67536e47b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.260105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7c55-3251-426a-89ed-37d67536e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.260576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7c55-3251-426a-89ed-37d67536e47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.261706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f7c55-3251-426a-89ed-37d67536e47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.263359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f7c55-3251-426a-89ed-37d67536e47b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.272280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-59689c52-64ad-4f0b-8ed7-5ae4bcf0d3ac None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:43.273844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-59689c52-64ad-4f0b-8ed7-5ae4bcf0d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:43.276554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 811/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:43.281561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0b329-4da5-4952-9c3b-8aa8e693cdc7 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.281849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0b329-4da5-4952-9c3b-8aa8e693cdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.282125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0b329-4da5-4952-9c3b-8aa8e693cdc7 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.283534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 812/1602] 10.222.0.17 () {54 vars in 962 bytes} [Sat Aug 6 14:29:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:29:43.287140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 791/1603] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:43.287281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0b329-4da5-4952-9c3b-8aa8e693cdc7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.287395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59689c52-64ad-4f0b-8ed7-5ae4bcf0d3ac None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.300354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0b329-4da5-4952-9c3b-8aa8e693cdc7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.304613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59689c52-64ad-4f0b-8ed7-5ae4bcf0d3ac None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.307230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.307986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.308336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.309039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] PATH_INFO: `/v3/projects/761f9cf9c26e4d80ae8c2471b3f79d5b/users/2dccf4227b054cb38cee4301cff80b17/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.316492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.317453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.317946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.318405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] PATH_INFO: `/v3/projects/96a7bb12d3ed4d1db2a58a03472483c2/users/5378c85cd31c42dfbc39ce738a8b6dde/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.326346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=761f9cf9c26e4d80ae8c2471b3f79d5b, user_id=2dccf4227b054cb38cee4301cff80b17, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.329379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.337421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a7bb12d3ed4d1db2a58a03472483c2, user_id=5378c85cd31c42dfbc39ce738a8b6dde, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.338574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:29:43.338574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:29:43.338574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:29:43.341590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.349537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 813/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:43 2022] PUT /identity/v3/projects/761f9cf9c26e4d80ae8c2471b3f79d5b/users/2dccf4227b054cb38cee4301cff80b17/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:43.350891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.367522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 792/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:43 2022] PUT /identity/v3/projects/96a7bb12d3ed4d1db2a58a03472483c2/users/5378c85cd31c42dfbc39ce738a8b6dde/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:43.369048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.426707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d876dbd-6ebc-4b85-bf4c-6d539d1f9bfd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.429389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:43.456551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9f696-da60-4e0b-acdf-a2e37fe8074c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.469997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.471891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.472661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.473114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] PATH_INFO: `/v3/projects/cd365cea868b4d9f8b9445569fd1e470/users/415ed747eb09431f87f85ed6be529920/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.496982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd365cea868b4d9f8b9445569fd1e470, user_id=415ed747eb09431f87f85ed6be529920, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.500479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.507622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6d14e-82a2-4ada-a58b-c7d8c0a8f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.508806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6d14e-82a2-4ada-a58b-c7d8c0a8f2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.509039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6d14e-82a2-4ada-a58b-c7d8c0a8f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.509469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6d14e-82a2-4ada-a58b-c7d8c0a8f2ff admin admin] PATH_INFO: `/v3/projects/c17d83661330425d8c75a2562df8e16e/users/a3e987dfd35f4a5da36d5034970fcf59/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.536984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 793/1606] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:43 2022] PUT /identity/v3/projects/cd365cea868b4d9f8b9445569fd1e470/users/415ed747eb09431f87f85ed6be529920/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:43.536984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.548200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6d14e-82a2-4ada-a58b-c7d8c0a8f2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c17d83661330425d8c75a2562df8e16e, user_id=a3e987dfd35f4a5da36d5034970fcf59, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.552601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6d14e-82a2-4ada-a58b-c7d8c0a8f2ff admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.556195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:43.575140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 814/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:43 2022] PUT /identity/v3/projects/c17d83661330425d8c75a2562df8e16e/users/a3e987dfd35f4a5da36d5034970fcf59/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:43.578147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.579083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.579357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.630667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:43.631972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:43.632631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bea56-049a-4fdd-b782-7855a978f390 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.728464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 815/1608] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Aug 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:43.729641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:43.730974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:43.736904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393b902-cc94-411a-a03c-40c05df00b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.738229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b902-cc94-411a-a03c-40c05df00b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.738540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b902-cc94-411a-a03c-40c05df00b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.739214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b902-cc94-411a-a03c-40c05df00b90 admin admin] PATH_INFO: `/v3/projects/f7ead8c81ff3486683d21f671acac6e0/users/8a1e8506e8b34a8d9c59b4acdbf23bb8/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.759216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393b902-cc94-411a-a03c-40c05df00b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ead8c81ff3486683d21f671acac6e0, user_id=8a1e8506e8b34a8d9c59b4acdbf23bb8, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:43.762736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393b902-cc94-411a-a03c-40c05df00b90 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:43.787816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 794/1609] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:43 2022] PUT /identity/v3/projects/f7ead8c81ff3486683d21f671acac6e0/users/8a1e8506e8b34a8d9c59b4acdbf23bb8/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:43.790975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f4d4f-8259-4e20-84ff-584d39d7db7f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.790975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f4d4f-8259-4e20-84ff-584d39d7db7f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.791214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f4d4f-8259-4e20-84ff-584d39d7db7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.806353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868a32a-6e8b-40fc-99d7-ebd3171ee5c4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:43.812830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cd0f4d4f-8259-4e20-84ff-584d39d7db7f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:43.814685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cd0f4d4f-8259-4e20-84ff-584d39d7db7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:43.871589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ea915-7727-4e4b-b1a9-8cb17b0da9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:43.872314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea915-7727-4e4b-b1a9-8cb17b0da9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.872682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea915-7727-4e4b-b1a9-8cb17b0da9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.873060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea915-7727-4e4b-b1a9-8cb17b0da9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.878976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:43.893073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 795/1610] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:43.895112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42633129-1903-4540-8118-9c30f08c0d12 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.895727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42633129-1903-4540-8118-9c30f08c0d12 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.896238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42633129-1903-4540-8118-9c30f08c0d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.930617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-42633129-1903-4540-8118-9c30f08c0d12 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:43.935121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-42633129-1903-4540-8118-9c30f08c0d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:43.949731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 796/1611] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:43.951494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c5a2a-690e-4d7c-aa5b-1be6889c44f3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:43.951774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c5a2a-690e-4d7c-aa5b-1be6889c44f3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:43.952049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c5a2a-690e-4d7c-aa5b-1be6889c44f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:43.982137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:43.989132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-697c5a2a-690e-4d7c-aa5b-1be6889c44f3 None None] MFA Rules not processed for user `415ed747eb09431f87f85ed6be529920`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:43.990702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-697c5a2a-690e-4d7c-aa5b-1be6889c44f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:44.013040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ea915-7727-4e4b-b1a9-8cb17b0da9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:44.014761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ea915-7727-4e4b-b1a9-8cb17b0da9e6 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:44.026691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 816/1612] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:44.029007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9d7e-ef78-4328-bd41-a13c2d7d0700 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.029595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9d7e-ef78-4328-bd41-a13c2d7d0700 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.030064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9d7e-ef78-4328-bd41-a13c2d7d0700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.050130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e7eb9d7e-ef78-4328-bd41-a13c2d7d0700 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:44.051395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e7eb9d7e-ef78-4328-bd41-a13c2d7d0700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:44.063137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 817/1613] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:44.063137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb9d7e-ef78-4328-bd41-a13c2d7d0700 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:44.080274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 797/1614] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:44.085739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797056a-21f4-4ea5-bafa-4d57fc98fd76 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.085739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797056a-21f4-4ea5-bafa-4d57fc98fd76 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.085739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797056a-21f4-4ea5-bafa-4d57fc98fd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.138537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6797056a-21f4-4ea5-bafa-4d57fc98fd76 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:44.140175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6797056a-21f4-4ea5-bafa-4d57fc98fd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:44.152127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 798/1615] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:44.153358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797056a-21f4-4ea5-bafa-4d57fc98fd76 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:44.164070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb9d7e-ef78-4328-bd41-a13c2d7d0700 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:44.171868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:44.172693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.172988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.173494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.218239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797056a-21f4-4ea5-bafa-4d57fc98fd76 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:44.232347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:44.232347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.232347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.232347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.245913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:44.250189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:44.280997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 799/1616] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:44.282340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:44.301336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986b9f2-268a-42ac-bc24-383a160cbde1 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:44.311249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9eaff5-1e99-4fcd-b2c4-d5451ad5ca2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:44.312026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eaff5-1e99-4fcd-b2c4-d5451ad5ca2f service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.312640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eaff5-1e99-4fcd-b2c4-d5451ad5ca2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.312992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eaff5-1e99-4fcd-b2c4-d5451ad5ca2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.324029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:44.327022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:44.327194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9eaff5-1e99-4fcd-b2c4-d5451ad5ca2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:44.328717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9eaff5-1e99-4fcd-b2c4-d5451ad5ca2f service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:44.342482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 818/1617] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:44.344135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:44.361288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e594f-ece4-47b7-acfa-4f31ad3c4fda service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:44.369406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185419ab-a812-41ac-8680-1fc8c6505a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:44.370208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185419ab-a812-41ac-8680-1fc8c6505a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.370605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185419ab-a812-41ac-8680-1fc8c6505a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.370993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185419ab-a812-41ac-8680-1fc8c6505a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.377156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 800/1618] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:44.383086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5ea1-a53d-4767-94b1-3c824bd9d440 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.383474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5ea1-a53d-4767-94b1-3c824bd9d440 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.384370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5ea1-a53d-4767-94b1-3c824bd9d440 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.387248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185419ab-a812-41ac-8680-1fc8c6505a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:44.389396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185419ab-a812-41ac-8680-1fc8c6505a95 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:44.390033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 801/1619] 10.222.0.17 () {54 vars in 962 bytes} [Sat Aug 6 14:29:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:29:44.391641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5ea1-a53d-4767-94b1-3c824bd9d440 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:44.407216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5ea1-a53d-4767-94b1-3c824bd9d440 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:44.413418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:44.414363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.414708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.415111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.419547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 819/1620] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:44.424657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf5bf-b9b6-4b2a-95cf-2b61697089fa None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.425762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf5bf-b9b6-4b2a-95cf-2b61697089fa None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.426127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cf5bf-b9b6-4b2a-95cf-2b61697089fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.430136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:44.432606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:44.453294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 802/1621] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:44.463490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-368cf5bf-b9b6-4b2a-95cf-2b61697089fa None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:44.465063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-368cf5bf-b9b6-4b2a-95cf-2b61697089fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:44.477298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 820/1622] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Aug 6 14:29:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:44.486691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:44.502685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586388a4-ee73-4336-ab07-f9e41f7697dc service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:44.513137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def57f9e-cc19-4852-9aa7-ba56692979ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:44.514110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def57f9e-cc19-4852-9aa7-ba56692979ed service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.514657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def57f9e-cc19-4852-9aa7-ba56692979ed service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.515477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def57f9e-cc19-4852-9aa7-ba56692979ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.527866 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def57f9e-cc19-4852-9aa7-ba56692979ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:44.529896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def57f9e-cc19-4852-9aa7-ba56692979ed service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:44.542595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 803/1623] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:44.728941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48020-b3a7-482d-bfed-be11e7ae8780 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.729504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48020-b3a7-482d-bfed-be11e7ae8780 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.729976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48020-b3a7-482d-bfed-be11e7ae8780 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.732297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 821/1624] 10.222.0.17 () {54 vars in 912 bytes} [Sat Aug 6 14:29:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:29:44.739181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aeddf-c6a5-4943-8515-f5a80e0a8ad7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:44.739604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aeddf-c6a5-4943-8515-f5a80e0a8ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:44.740003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aeddf-c6a5-4943-8515-f5a80e0a8ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:44.814744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-673aeddf-c6a5-4943-8515-f5a80e0a8ad7 None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:44.818263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-673aeddf-c6a5-4943-8515-f5a80e0a8ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:44.849791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 804/1625] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:29:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:29:44.861846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d48020-b3a7-482d-bfed-be11e7ae8780 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:44.978028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:44.989494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d48020-b3a7-482d-bfed-be11e7ae8780 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:44.998958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a542f7-4fc2-4013-8d13-053e44ab263c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:44.999778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a542f7-4fc2-4013-8d13-053e44ab263c service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:45.000198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a542f7-4fc2-4013-8d13-053e44ab263c service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:45.001039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a542f7-4fc2-4013-8d13-053e44ab263c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:45.027818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a542f7-4fc2-4013-8d13-053e44ab263c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:45.029528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a542f7-4fc2-4013-8d13-053e44ab263c service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:45.051066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 822/1626] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:45.093466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224b3c2-5b4a-4125-8725-5e4ea7eb6a77 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:45.094128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224b3c2-5b4a-4125-8725-5e4ea7eb6a77 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:45.094128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224b3c2-5b4a-4125-8725-5e4ea7eb6a77 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:45.096895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 805/1627] 10.222.0.17 () {54 vars in 912 bytes} [Sat Aug 6 14:29:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:29:45.104778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b661990-1b98-4430-873e-32519f00dfb1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:45.105377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b661990-1b98-4430-873e-32519f00dfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:45.105819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b661990-1b98-4430-873e-32519f00dfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:45.136273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5b661990-1b98-4430-873e-32519f00dfb1 None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:45.137497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5b661990-1b98-4430-873e-32519f00dfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:45.150176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 823/1628] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:29:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:29:45.167826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224b3c2-5b4a-4125-8725-5e4ea7eb6a77 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:45.183627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224b3c2-5b4a-4125-8725-5e4ea7eb6a77 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:45.194724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809d026-9a34-4d41-8411-ddb8562daab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:45.203917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809d026-9a34-4d41-8411-ddb8562daab7 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:45.204439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809d026-9a34-4d41-8411-ddb8562daab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:45.204736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809d026-9a34-4d41-8411-ddb8562daab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:45.228676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809d026-9a34-4d41-8411-ddb8562daab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:45.230328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809d026-9a34-4d41-8411-ddb8562daab7 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:45.245342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 806/1629] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:45.538827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b661990-1b98-4430-873e-32519f00dfb1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:45.561034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b661990-1b98-4430-873e-32519f00dfb1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:45.573795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:45.574462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:45.574746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:45.575068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] PATH_INFO: `/v3/users/415ed747eb09431f87f85ed6be529920` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:45.576520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415ed747eb09431f87f85ed6be529920)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:45.578237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:45.687463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415ed747eb09431f87f85ed6be529920'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:45.694773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415ed747eb09431f87f85ed6be529920'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:45.698628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415ed747eb09431f87f85ed6be529920'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:45.729589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415ed747eb09431f87f85ed6be529920'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:45.746619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415ed747eb09431f87f85ed6be529920'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:45.773330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 824/1630] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:29:45 2022] DELETE /identity/v3/users/415ed747eb09431f87f85ed6be529920 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:45.782181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0498fe-c985-440f-815a-47c721b11487 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:45.782494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0498fe-c985-440f-815a-47c721b11487 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:45.782734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0498fe-c985-440f-815a-47c721b11487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:45.824865 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4d0498fe-c985-440f-815a-47c721b11487 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:45.829064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4d0498fe-c985-440f-815a-47c721b11487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:46.011398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 807/1631] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:46.051786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:46.053820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:46.267523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10df0f6-25c4-4996-b13e-56666fc46a04 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:46.278014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:46.280569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:46.280843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:46.281121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:46.336662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:46.340576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:46.374259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 825/1632] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:29:46.601738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe2d2-8c37-43c7-8ae5-6442640e71dd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:46.603994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe2d2-8c37-43c7-8ae5-6442640e71dd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:46.604542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe2d2-8c37-43c7-8ae5-6442640e71dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:46.624749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:46.677869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-43fbe2d2-8c37-43c7-8ae5-6442640e71dd None None] MFA Rules not processed for user `2dccf4227b054cb38cee4301cff80b17`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:46.683013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-43fbe2d2-8c37-43c7-8ae5-6442640e71dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:46.903481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 808/1633] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:46 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:46.905981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:46.907843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:46.930517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678f45-43b9-4454-b998-be7b2dec57b9 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:46.942608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:46.943377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:46.943661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:46.943942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:46.944680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:46.947620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:46.996519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 826/1634] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:46 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:46.997827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe2d2-8c37-43c7-8ae5-6442640e71dd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:47.011294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe2d2-8c37-43c7-8ae5-6442640e71dd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:47.113589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:47.116590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:47.116875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:47.117153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:47.120538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:47.125701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:47.193394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 809/1635] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:46 2022] POST /identity/v3/users => generated 309 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:47.194984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:47.211439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9456eaa-189c-4dbd-a520-ae938487690d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:47.224127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:47.225779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:47.226690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:47.227007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:47.230451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:47.237297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:47.263296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 827/1636] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:47.267801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:47.283362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8811f-29f7-41d2-bf44-9d45da31d059 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:47.298172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:47.299103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:47.299388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:47.299667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] PATH_INFO: `/v3/projects/9ea0568500c146179e2034a34566a28d/users/7da1266c93f5442193b10c1146566be7/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:47.341455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea0568500c146179e2034a34566a28d, user_id=7da1266c93f5442193b10c1146566be7, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:47.355877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:47.379561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 810/1637] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:47 2022] PUT /identity/v3/projects/9ea0568500c146179e2034a34566a28d/users/7da1266c93f5442193b10c1146566be7/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:47.381122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:47.397720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19da03-4b7f-4f0e-ae9d-daf8b9c50c39 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:47.465515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:47.564212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:47.567413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:47.569831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:47.569976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:47.570727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:47.574472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:47.602986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 828/1638] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:47.605727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:47.621337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0eea0-152b-487f-8ff5-35f094966b06 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:47.635833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e498-3a58-426d-b5c5-e7dc176ae820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:47.636555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e498-3a58-426d-b5c5-e7dc176ae820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:47.636839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e498-3a58-426d-b5c5-e7dc176ae820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:47.637131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e498-3a58-426d-b5c5-e7dc176ae820 admin admin] PATH_INFO: `/v3/projects/9ea0568500c146179e2034a34566a28d/users/7da1266c93f5442193b10c1146566be7/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:47.642184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e498-3a58-426d-b5c5-e7dc176ae820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea0568500c146179e2034a34566a28d, user_id=7da1266c93f5442193b10c1146566be7, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:47.644913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e498-3a58-426d-b5c5-e7dc176ae820 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:47.676000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 811/1639] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:47 2022] PUT /identity/v3/projects/9ea0568500c146179e2034a34566a28d/users/7da1266c93f5442193b10c1146566be7/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:48.221215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:48.223869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:48.247043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eaef6-962c-4768-b0ad-f54f9803324f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:48.422413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:48.422898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:48.423284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:48.423637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] PATH_INFO: `/v3/projects/cd365cea868b4d9f8b9445569fd1e470` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:48.425552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd365cea868b4d9f8b9445569fd1e470)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:48.427755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:29:48.427755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Aug 06 14:29:48.427755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: Aug 06 14:29:48.431966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:48.623649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd365cea868b4d9f8b9445569fd1e470', 'request_id': 'req-96fb11c0-b173-4212-a0da-63e13f9696c5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:48.664690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd365cea868b4d9f8b9445569fd1e470', 'request_id': 'req-96fb11c0-b173-4212-a0da-63e13f9696c5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:48.682802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd365cea868b4d9f8b9445569fd1e470', 'request_id': 'req-96fb11c0-b173-4212-a0da-63e13f9696c5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:48.711755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] The token cache is being invalidate because project cd365cea868b4d9f8b9445569fd1e470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:29:48.712054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:48.713497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-96fb11c0-b173-4212-a0da-63e13f9696c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:48.716457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 829/1640] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:29:48 2022] DELETE /identity/v3/projects/cd365cea868b4d9f8b9445569fd1e470 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:48.722268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4cb4-5ee4-4e95-a344-b957ef7e8d6e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:48.723273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4cb4-5ee4-4e95-a344-b957ef7e8d6e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:48.723722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4cb4-5ee4-4e95-a344-b957ef7e8d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:48.780412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f78f4cb4-5ee4-4e95-a344-b957ef7e8d6e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:48.784110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f78f4cb4-5ee4-4e95-a344-b957ef7e8d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:48.889867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:48.930107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 812/1641] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:48.938079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a044f30-94bf-4ddc-96b1-9c843a593691 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:48.938556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a044f30-94bf-4ddc-96b1-9c843a593691 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:48.938982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a044f30-94bf-4ddc-96b1-9c843a593691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:48.978998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5a044f30-94bf-4ddc-96b1-9c843a593691 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:48.984205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5a044f30-94bf-4ddc-96b1-9c843a593691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:49.018162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4cb4-5ee4-4e95-a344-b957ef7e8d6e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:49.035472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 830/1642] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:49.039467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4cb4-5ee4-4e95-a344-b957ef7e8d6e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:49.061022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:49.061682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:49.062051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:49.062228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:49.062974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:49.063289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:49.064929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:49.102263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 813/1643] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:29:49.106804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a044f30-94bf-4ddc-96b1-9c843a593691 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:49.125582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a044f30-94bf-4ddc-96b1-9c843a593691 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:49.148125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:49.149692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:49.150095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:49.150579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:49.152647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:49.155715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:49.228678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:49.232411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 831/1644] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:49 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:49.247831 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ea8c-bdef-4956-9155-a99ea48625f8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:49.489194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:49.489812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:49.490128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:49.490406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:49.503830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:49.505839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:49.551122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 814/1645] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:49 2022] POST /identity/v3/users => generated 289 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:49.556327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:49.568645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0261-5123-4041-bdd3-1913b8183d1f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:49.590858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:49.591585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:49.591878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:49.592163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:49.592887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:49.603358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:49.627607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 832/1646] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:49.632376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:49.661599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbc5cf-5d87-4bf6-b771-b18711b278c1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:49.672588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e18bb-418c-4542-ac97-0b85b0b61bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:49.673285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e18bb-418c-4542-ac97-0b85b0b61bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:49.673577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e18bb-418c-4542-ac97-0b85b0b61bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:49.676629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e18bb-418c-4542-ac97-0b85b0b61bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:49.677805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e18bb-418c-4542-ac97-0b85b0b61bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:49.680184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e18bb-418c-4542-ac97-0b85b0b61bc7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:49.703286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:49.714409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 815/1647] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:49.717592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2ab52-af7b-45fd-b9d0-89823c72ce85 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:49.733650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:49.734662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:49.735325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:49.735804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] PATH_INFO: `/v3/projects/0efbe1d6f7094a8e9ded71857a5819cf/users/a9680a06ed9b4ac39a9793ae55d133e1/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:49.766076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:49.776443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efbe1d6f7094a8e9ded71857a5819cf, user_id=a9680a06ed9b4ac39a9793ae55d133e1, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:49.784609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:49.827804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaac18-0bf0-41d8-9eff-167b34381c50 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:49.828339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaac18-0bf0-41d8-9eff-167b34381c50 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:49.828339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaac18-0bf0-41d8-9eff-167b34381c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:49.833904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 833/1648] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:49 2022] PUT /identity/v3/projects/0efbe1d6f7094a8e9ded71857a5819cf/users/a9680a06ed9b4ac39a9793ae55d133e1/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:49.902706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9aeaac18-0bf0-41d8-9eff-167b34381c50 None None] MFA Rules not processed for user `a9680a06ed9b4ac39a9793ae55d133e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:49.904086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9aeaac18-0bf0-41d8-9eff-167b34381c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:50.080841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:50.083009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 816/1649] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:50.212852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd2631-f971-4878-9f4e-2bd9ee1aaa5c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:50.233131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f383d77-1eaf-4bec-ac8f-5c3283967c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:50.234705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f383d77-1eaf-4bec-ac8f-5c3283967c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.235307 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f383d77-1eaf-4bec-ac8f-5c3283967c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.235826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f383d77-1eaf-4bec-ac8f-5c3283967c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.238105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f383d77-1eaf-4bec-ac8f-5c3283967c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:50.241179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f383d77-1eaf-4bec-ac8f-5c3283967c0d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:50.289210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 834/1650] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:50 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:50.293597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaac18-0bf0-41d8-9eff-167b34381c50 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:50.310591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaac18-0bf0-41d8-9eff-167b34381c50 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:50.362764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228df07-13ec-45da-8203-2b6e930a95f1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.363241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228df07-13ec-45da-8203-2b6e930a95f1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.364306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228df07-13ec-45da-8203-2b6e930a95f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.389204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:50.409761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3228df07-13ec-45da-8203-2b6e930a95f1 None None] MFA Rules not processed for user `4dadf737392d4153ae50de020534f070`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:50.411410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:50.412355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.412607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3228df07-13ec-45da-8203-2b6e930a95f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:50.412712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.413064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.413883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:50.417738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:50.446681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 817/1651] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:50 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:50.449986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:50.463508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020fdc9-c8f2-4db7-a5db-dc426e7824eb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:50.471743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:50.472910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.472910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.472910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.473824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:50.476834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:50.499890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 818/1652] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:50.503486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:50.521744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 835/1653] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:50.521744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.523495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3e16c-338a-4c2a-842a-f5fe732ae296 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:50.523679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.524204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.538158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:50.538699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.539100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.539472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.540369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:50.542081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:50.554590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 819/1654] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:50.555247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] MFA Rules not processed for user `4dadf737392d4153ae50de020534f070`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:50.557831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:50.561876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:50.576853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 836/1655] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:50.581267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7373b2-2e03-42af-b189-f2440f116e53 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:50.584230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:50.586345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:50.587019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4653e-d5d3-4d83-989f-131cf72c80cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:50.587762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4653e-d5d3-4d83-989f-131cf72c80cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.588161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4653e-d5d3-4d83-989f-131cf72c80cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.588532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4653e-d5d3-4d83-989f-131cf72c80cd admin admin] PATH_INFO: `/v3/projects/9c75694ba7ce4d5183311f6519a14cf4/users/1cdbf6de97c14d1b85d4e076f64fa223/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.613177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4653e-d5d3-4d83-989f-131cf72c80cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c75694ba7ce4d5183311f6519a14cf4, user_id=1cdbf6de97c14d1b85d4e076f64fa223, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:50.616433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4653e-d5d3-4d83-989f-131cf72c80cd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:50.648611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 820/1656] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:50 2022] PUT /identity/v3/projects/9c75694ba7ce4d5183311f6519a14cf4/users/1cdbf6de97c14d1b85d4e076f64fa223/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:50.660461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0808f7-e055-44b8-8978-4f68b66e0b7b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.660783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0808f7-e055-44b8-8978-4f68b66e0b7b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.661064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0808f7-e055-44b8-8978-4f68b66e0b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.668697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:50.693431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c5c6-08d5-4cf8-a7b2-be1e25f6b18b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:50.713848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ce0808f7-e055-44b8-8978-4f68b66e0b7b None None] MFA Rules not processed for user `1cdbf6de97c14d1b85d4e076f64fa223`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:50.718424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ce0808f7-e055-44b8-8978-4f68b66e0b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:50.821873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 821/1657] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:29:50.826022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0808f7-e055-44b8-8978-4f68b66e0b7b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:50.837182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:50.837885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:50.838398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:50.838819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:50.972802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:50.974937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:50.996351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 837/1658] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:50.999705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0808f7-e055-44b8-8978-4f68b66e0b7b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:51.008277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:51.008971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.009256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.009535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.010219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:51.013999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:51.045678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:51.063231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 822/1659] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:50 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:29:51.071588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:51.103777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0b32d-dbb3-4d8a-be8e-51fddc182e1f service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:51.262053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:51.262713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.263030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.263305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.263987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:51.266468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:51.303364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 838/1660] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:51 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:51.307406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:51.322458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f0493-d2fd-4070-be92-b86fc19dc2b3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:51.330776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:51.331780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.332694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.333191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.334099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:51.336266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:51.374599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 823/1661] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:51.378712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:51.389097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8b8d-9cc6-432c-8ce8-52973937e270 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:51.395299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:51.395873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.396406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.396508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] PATH_INFO: `/v3/projects/a2b14fbcc21b45988a37758de43d061e/users/8c2bfdd87f344309ba264a0aeabab3c8/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.424249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b14fbcc21b45988a37758de43d061e, user_id=8c2bfdd87f344309ba264a0aeabab3c8, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:51.427517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:51.445595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:51.460541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:51.463004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 839/1662] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:51 2022] PUT /identity/v3/projects/a2b14fbcc21b45988a37758de43d061e/users/8c2bfdd87f344309ba264a0aeabab3c8/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:51.478280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27a3e-50c1-4bb4-8d36-88af37b6a9c4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:51.605771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d6e56-2758-437a-ada7-1944a5c87f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:51.606958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d6e56-2758-437a-ada7-1944a5c87f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.607249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d6e56-2758-437a-ada7-1944a5c87f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.607679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d6e56-2758-437a-ada7-1944a5c87f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.609222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d6e56-2758-437a-ada7-1944a5c87f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:51.612495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d6e56-2758-437a-ada7-1944a5c87f4e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:51.645037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:51.657349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 824/1663] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:51.659782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c406a-cba9-4a9f-871a-23a34d6c95ba admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:51.669085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b61901-91e8-4f7e-8b94-0bfe81648240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:51.670020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61901-91e8-4f7e-8b94-0bfe81648240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.670400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61901-91e8-4f7e-8b94-0bfe81648240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.670767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61901-91e8-4f7e-8b94-0bfe81648240 admin admin] PATH_INFO: `/v3/projects/a2b14fbcc21b45988a37758de43d061e/users/8c2bfdd87f344309ba264a0aeabab3c8/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.676017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b61901-91e8-4f7e-8b94-0bfe81648240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b14fbcc21b45988a37758de43d061e, user_id=8c2bfdd87f344309ba264a0aeabab3c8, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:51.679785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b61901-91e8-4f7e-8b94-0bfe81648240 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:51.710423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.710423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.710423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.719855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 840/1664] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:51 2022] PUT /identity/v3/projects/a2b14fbcc21b45988a37758de43d061e/users/8c2bfdd87f344309ba264a0aeabab3c8/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:51.761508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] MFA Rules not processed for user `8c2bfdd87f344309ba264a0aeabab3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:51.762785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:51.782480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af74d-1026-4c7f-b9ca-aeb94d176919 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:51.782784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af74d-1026-4c7f-b9ca-aeb94d176919 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:51.785369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af74d-1026-4c7f-b9ca-aeb94d176919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:51.873228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0d4af74d-1026-4c7f-b9ca-aeb94d176919 None None] MFA Rules not processed for user `7da1266c93f5442193b10c1146566be7`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:51.877822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0d4af74d-1026-4c7f-b9ca-aeb94d176919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:51.927731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 825/1665] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:51.940110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:51.941814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:52.023873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6a37-476d-4562-83ca-cb05257658f8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:52.028665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:52.029447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:52.029791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:52.030153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:52.042569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:52.044177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:52.063305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 826/1666] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:52.104891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 841/1667] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:52.145847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:52.148081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:52.162166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b5984-9821-4c67-af8b-9d4b37e60276 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:52.167046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe67a00-2716-417f-ad08-ad949c913524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:52.167738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe67a00-2716-417f-ad08-ad949c913524 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:52.168084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe67a00-2716-417f-ad08-ad949c913524 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:52.168425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe67a00-2716-417f-ad08-ad949c913524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:52.185305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:52.289576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe67a00-2716-417f-ad08-ad949c913524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:52.292587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe67a00-2716-417f-ad08-ad949c913524 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:52.312857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 827/1668] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:29:52.378044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81f5c-054c-4c92-a562-4d16088aa05f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:52.379013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81f5c-054c-4c92-a562-4d16088aa05f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:52.379551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81f5c-054c-4c92-a562-4d16088aa05f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:52.384233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 842/1669] 10.222.0.17 () {54 vars in 912 bytes} [Sat Aug 6 14:29:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:29:52.405670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66b979-b0f2-4093-91c5-96d172d39070 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:52.406418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66b979-b0f2-4093-91c5-96d172d39070 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:52.407040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66b979-b0f2-4093-91c5-96d172d39070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:52.480983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4f66b979-b0f2-4093-91c5-96d172d39070 None None] MFA Rules not processed for user `cd235744a7ee462eb13e383930de4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:52.485658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4f66b979-b0f2-4093-91c5-96d172d39070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:52.596347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 828/1670] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:29:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:52.920312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81f5c-054c-4c92-a562-4d16088aa05f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:52.921901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5ab81f5c-054c-4c92-a562-4d16088aa05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:52.950965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81f5c-054c-4c92-a562-4d16088aa05f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:52.959935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee16d3-37e3-409b-8a28-eb7fe2eae4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:52.964077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee16d3-37e3-409b-8a28-eb7fe2eae4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:52.964363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee16d3-37e3-409b-8a28-eb7fe2eae4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:52.973201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee16d3-37e3-409b-8a28-eb7fe2eae4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:52.996958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee16d3-37e3-409b-8a28-eb7fe2eae4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:53.001775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee16d3-37e3-409b-8a28-eb7fe2eae4c4 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:53.053035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 843/1671] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:53.378439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef225af9-05e9-4703-9c64-4994656cd5f1 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:53.379019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef225af9-05e9-4703-9c64-4994656cd5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:53.379344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef225af9-05e9-4703-9c64-4994656cd5f1 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:53.380954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 829/1672] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:29:53.402324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:53.402818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:53.403064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:53.440571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:53.441813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:53.636457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 844/1673] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:29:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:53.671265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef225af9-05e9-4703-9c64-4994656cd5f1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:53.673453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ef225af9-05e9-4703-9c64-4994656cd5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:53.695762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef225af9-05e9-4703-9c64-4994656cd5f1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:53.707459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:53.708709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:53.709050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:53.709327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:53.720592 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:53.722130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:53.751143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 830/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:54.321495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:54.323578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:54.428686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:54.464113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb900dd-3224-4591-943d-8c1664f1593f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:54.481396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:54.482836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:54.483594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:54.484105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] PATH_INFO: `/v3/users/2dccf4227b054cb38cee4301cff80b17` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:54.486225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dccf4227b054cb38cee4301cff80b17)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:54.489157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:54.564923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dccf4227b054cb38cee4301cff80b17'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:54.565852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dccf4227b054cb38cee4301cff80b17'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:54.566625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dccf4227b054cb38cee4301cff80b17'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:54.584209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dccf4227b054cb38cee4301cff80b17'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:54.602803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-18414b45-5043-4ac7-81c0-99d2a6d6f6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dccf4227b054cb38cee4301cff80b17'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:54.621799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:54.626776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 845/1675] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:29:54 2022] DELETE /identity/v3/users/2dccf4227b054cb38cee4301cff80b17 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:54.637140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ed80f-cb82-43fb-9910-17449bc53535 service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:54.703270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f5098-aef3-4ac3-b251-b5f691a27a60 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:54.703715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f5098-aef3-4ac3-b251-b5f691a27a60 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:54.704111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f5098-aef3-4ac3-b251-b5f691a27a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:54.737950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fd4f5098-aef3-4ac3-b251-b5f691a27a60 None None] MFA Rules not processed for user `4dadf737392d4153ae50de020534f070`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:54.740641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:54.741287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:54.742962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:54.743442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:54.743897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fd4f5098-aef3-4ac3-b251-b5f691a27a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:54.745635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:54.758845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:54.763987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:54.802972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 831/1676] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:29:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:29:54.891616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 846/1677] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:29:54.952669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:54.954075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:55.064768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:55.207610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a528ec-feeb-4083-87a1-f946c457831f service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:55.227601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:55.230531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:55.230855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:55.234029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] PATH_INFO: `/v3/projects/761f9cf9c26e4d80ae8c2471b3f79d5b` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:55.239630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=761f9cf9c26e4d80ae8c2471b3f79d5b)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:55.244679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:55.381728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761f9cf9c26e4d80ae8c2471b3f79d5b', 'request_id': 'req-014f9ab7-b560-4093-a322-89bed9f15460'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.393279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761f9cf9c26e4d80ae8c2471b3f79d5b', 'request_id': 'req-014f9ab7-b560-4093-a322-89bed9f15460'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.420914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761f9cf9c26e4d80ae8c2471b3f79d5b', 'request_id': 'req-014f9ab7-b560-4093-a322-89bed9f15460'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.446737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] The token cache is being invalidate because project 761f9cf9c26e4d80ae8c2471b3f79d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:29:55.451284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.452483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.457784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 832/1678] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:29:54 2022] DELETE /identity/v3/projects/761f9cf9c26e4d80ae8c2471b3f79d5b => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:55.459278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f5098-aef3-4ac3-b251-b5f691a27a60 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:55.483637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f5098-aef3-4ac3-b251-b5f691a27a60 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:55.590141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:55.592063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:55.592352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:55.593795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] PATH_INFO: `/v3/users/7da1266c93f5442193b10c1146566be7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:55.597581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da1266c93f5442193b10c1146566be7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:55.599569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:55.656854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:55.709017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da1266c93f5442193b10c1146566be7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.710625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da1266c93f5442193b10c1146566be7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.713320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da1266c93f5442193b10c1146566be7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.746923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da1266c93f5442193b10c1146566be7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.763525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fc398c3-8e8e-4383-a489-1d46f046b5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da1266c93f5442193b10c1146566be7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:55.816009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 847/1679] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:29:55 2022] DELETE /identity/v3/users/7da1266c93f5442193b10c1146566be7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:56.811889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:56.814532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:56.847620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f9ab7-b560-4093-a322-89bed9f15460 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:56.975566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:56.976487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:56.978063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:56.978449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] PATH_INFO: `/v3/projects/9ea0568500c146179e2034a34566a28d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:56.982104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea0568500c146179e2034a34566a28d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:56.990232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:57.113524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea0568500c146179e2034a34566a28d', 'request_id': 'req-089d1bf5-934b-4a36-b022-ab21065c4249'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:57.127415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea0568500c146179e2034a34566a28d', 'request_id': 'req-089d1bf5-934b-4a36-b022-ab21065c4249'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:57.133140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:57.147731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea0568500c146179e2034a34566a28d', 'request_id': 'req-089d1bf5-934b-4a36-b022-ab21065c4249'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:57.169143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] The token cache is being invalidate because project 9ea0568500c146179e2034a34566a28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:29:57.169441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:57.171586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-089d1bf5-934b-4a36-b022-ab21065c4249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:29:57.177563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 833/1680] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:29:56 2022] DELETE /identity/v3/projects/9ea0568500c146179e2034a34566a28d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:57.179068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaaa97a-3d37-4aac-ab6a-1b20a0978b14 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:57.180739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaaa97a-3d37-4aac-ab6a-1b20a0978b14 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:57.181378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaaa97a-3d37-4aac-ab6a-1b20a0978b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:57.207526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-abaaa97a-3d37-4aac-ab6a-1b20a0978b14 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:57.209734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-abaaa97a-3d37-4aac-ab6a-1b20a0978b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:57.311547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 848/1681] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:57.314336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50f34e-6793-472c-89a1-b2139860f122 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:57.314938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50f34e-6793-472c-89a1-b2139860f122 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:57.315431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50f34e-6793-472c-89a1-b2139860f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:57.371071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8c50f34e-6793-472c-89a1-b2139860f122 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:57.377345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8c50f34e-6793-472c-89a1-b2139860f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:57.408330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaaa97a-3d37-4aac-ab6a-1b20a0978b14 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:57.417281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 834/1682] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:57.425348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaaa97a-3d37-4aac-ab6a-1b20a0978b14 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:57.432313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:57.433179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:57.433601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:57.434003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:57.435100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:29:57.435556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:57.437560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:57.469183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 849/1683] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:29:57.474008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50f34e-6793-472c-89a1-b2139860f122 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:57.488517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50f34e-6793-472c-89a1-b2139860f122 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:57.512601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:57.513586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:57.516861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:57.517278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:57.521402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:57.546233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:57.631035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 835/1684] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:57 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:57.634530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:57.659091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fe205-654a-49e9-a32f-ca537547656c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:57.823119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:57.823762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:57.824047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:57.824324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:57.824998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:57.827159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:57.866601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 850/1685] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:57 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:57.870918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:57.884871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82a3aa-e19b-4f8d-85ca-11cbf24df9dd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:57.892731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:57.893903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:57.894385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:57.894894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:57.895975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:57.898267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:57.916502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 836/1686] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:57.917148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:57.943518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef3e0-2829-4348-88bf-7f8a1a339ba7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:57.953434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c459a3c-5ae6-42ad-83f5-9d0b6da648b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:57.954283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459a3c-5ae6-42ad-83f5-9d0b6da648b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:57.954674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459a3c-5ae6-42ad-83f5-9d0b6da648b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:57.955661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459a3c-5ae6-42ad-83f5-9d0b6da648b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:57.956508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c459a3c-5ae6-42ad-83f5-9d0b6da648b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:57.962474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c459a3c-5ae6-42ad-83f5-9d0b6da648b3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:58.013210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 851/1687] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:58.014490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:58.082105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca8656-2b91-4693-9987-bc3f0b761cbf admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:58.104492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:58.106707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:58.107449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:58.107884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] PATH_INFO: `/v3/projects/8b26c5e0805d440a806183cffd373f28/users/87733319a8404d67925cfc5c138055d9/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:58.164003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b26c5e0805d440a806183cffd373f28, user_id=87733319a8404d67925cfc5c138055d9, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:58.167393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:58.204081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 837/1688] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:58 2022] PUT /identity/v3/projects/8b26c5e0805d440a806183cffd373f28/users/87733319a8404d67925cfc5c138055d9/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:58.216792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed622aee-ed11-46f5-9aca-33624118fe64 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:58.217263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed622aee-ed11-46f5-9aca-33624118fe64 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:58.217473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed622aee-ed11-46f5-9aca-33624118fe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:58.230845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:58.306492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ed622aee-ed11-46f5-9aca-33624118fe64 None None] MFA Rules not processed for user `87733319a8404d67925cfc5c138055d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:58.312395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ed622aee-ed11-46f5-9aca-33624118fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:58.467925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 852/1689] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:58.469219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:58.584206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d1465-1410-4e3c-ad35-f5a3891693b9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:58.585088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d1465-1410-4e3c-ad35-f5a3891693b9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:58.585377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d1465-1410-4e3c-ad35-f5a3891693b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:58.645746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522c18-ab67-4979-9eb7-c151bcf437b9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:58.656561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:58.657566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:58.657566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:58.658399 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:58.660176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:58.661103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:58.743099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b43d1465-1410-4e3c-ad35-f5a3891693b9 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:58.745488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b43d1465-1410-4e3c-ad35-f5a3891693b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:58.754162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 838/1690] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:29:58 2022] POST /identity/v3/projects => generated 375 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:29:58.758211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:58.901688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69150012-e306-4103-87b6-48979cedc089 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:58.907024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 853/1691] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:29:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:58.907437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:58.908144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:58.912185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:58.912689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:58.913621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:58.917740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:58.960475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:58.995314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 839/1692] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:29:58 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:29:59.001024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d1465-1410-4e3c-ad35-f5a3891693b9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:59.023351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d1465-1410-4e3c-ad35-f5a3891693b9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:59.040982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:59.044973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:59.045796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:59.046089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:59.049470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:59.051561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:59.089966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 854/1693] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:59.091352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:59.113783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f9d2-d515-4e08-856e-4cc9ffa7d29e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:59.125527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:59.126392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:59.126760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:59.127256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] PATH_INFO: `/v3/projects/509074a028df4596b77698a521396fc0/users/7fa2fc9489504a23ab1f9f1748b18168/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:59.131489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:59.133379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:59.151157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509074a028df4596b77698a521396fc0, user_id=7fa2fc9489504a23ab1f9f1748b18168, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:59.154900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:59.185964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 840/1694] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:59 2022] PUT /identity/v3/projects/509074a028df4596b77698a521396fc0/users/7fa2fc9489504a23ab1f9f1748b18168/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:59.198692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:59.319873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:29:59.410152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b590460-eb18-4a89-b6c3-1633aa0af6b1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:59.427857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:59.428501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:59.428786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:59.429060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:59.429771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:59.432203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:59.449262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5165e2-1012-43a3-945c-d86af57c37ef admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:59.474652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 841/1695] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:29:59.478648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:59.499721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804be08-add2-429b-a25c-528a6df37bc1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:59.517546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308ed18-249d-40c1-acd2-304460a4a4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:59.518250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ed18-249d-40c1-acd2-304460a4a4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:59.518522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ed18-249d-40c1-acd2-304460a4a4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:59.518799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ed18-249d-40c1-acd2-304460a4a4a8 admin admin] PATH_INFO: `/v3/projects/509074a028df4596b77698a521396fc0/users/7fa2fc9489504a23ab1f9f1748b18168/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:59.521457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308ed18-249d-40c1-acd2-304460a4a4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509074a028df4596b77698a521396fc0, user_id=7fa2fc9489504a23ab1f9f1748b18168, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:59.524693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308ed18-249d-40c1-acd2-304460a4a4a8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:59.563399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:29:59.563686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:59.563964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:59.564243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:59.580642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 842/1696] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:29:59 2022] PUT /identity/v3/projects/509074a028df4596b77698a521396fc0/users/7fa2fc9489504a23ab1f9f1748b18168/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:29:59.591525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:29:59.591839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:29:59.592079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:29:59.692507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] MFA Rules not processed for user `7fa2fc9489504a23ab1f9f1748b18168`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:29:59.693732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:59.776388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:29:59.780189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:29:59.826546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 855/1697] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:29:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 14:29:59.828469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:59.831809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:29:59.872236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e8d4e-b14b-40c1-89ea-915ba4ea9b4b service swift] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:29:59.913486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 843/1698] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:29:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:29:59.914394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:29:59.926419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:00.075530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:00.076276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:00.076702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:00.077057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] PATH_INFO: `/v3/users/a9680a06ed9b4ac39a9793ae55d133e1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:00.078564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9680a06ed9b4ac39a9793ae55d133e1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:00.080753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:00.090795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:00.138362 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:00.162204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584a633-d43b-409e-b6af-af46cd14d6bb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:00.177655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:00.178614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:00.179161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:00.179658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] PATH_INFO: `/v3/users/a6275ec0936a4471a49b9a6aa1eef931` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:00.183460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6275ec0936a4471a49b9a6aa1eef931)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:00.184987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 14:30:00.184987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Aug 06 14:30:00.184987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: Aug 06 14:30:00.188198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:00.247508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9680a06ed9b4ac39a9793ae55d133e1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.249170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9680a06ed9b4ac39a9793ae55d133e1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.251621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9680a06ed9b4ac39a9793ae55d133e1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.304456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9680a06ed9b4ac39a9793ae55d133e1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.325524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9680a06ed9b4ac39a9793ae55d133e1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.354647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 856/1699] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:29:59 2022] DELETE /identity/v3/users/a9680a06ed9b4ac39a9793ae55d133e1 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:00.356187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:00.361844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:00.405346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6275ec0936a4471a49b9a6aa1eef931'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.421386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6275ec0936a4471a49b9a6aa1eef931'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.428392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6275ec0936a4471a49b9a6aa1eef931'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.437704 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6275ec0936a4471a49b9a6aa1eef931'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.467415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1494de25-f7f7-4763-8dcf-db49166e39fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6275ec0936a4471a49b9a6aa1eef931'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:00.512711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 844/1700] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:29:59 2022] DELETE /identity/v3/users/a6275ec0936a4471a49b9a6aa1eef931 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:00.514499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4f8f6-8c80-4e37-b426-002d24684eb3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:00.514979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4f8f6-8c80-4e37-b426-002d24684eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:00.515416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4f8f6-8c80-4e37-b426-002d24684eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:00.531876 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e559e8-7af6-4c47-887c-26f744d23c1c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:00.590956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3bb4f8f6-8c80-4e37-b426-002d24684eb3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:00.595071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3bb4f8f6-8c80-4e37-b426-002d24684eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:00.673167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:00.674220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:00.674621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:00.675031 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:00.680146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:00.773373 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 845/1701] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:00.775418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:00.775712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:00.776000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:00.835939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:00.839815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:00.871915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 846/1702] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:00.873932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:00.877996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:00.919314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 857/1703] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:00.987143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:00.991408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:01.001707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:01.003325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:01.021746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:01.168483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774000d-9690-41c1-8e8a-70366f6c4ee8 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:01.177814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb467d5-d32a-4911-9810-019fe28695e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:01.178600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb467d5-d32a-4911-9810-019fe28695e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:01.182967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb467d5-d32a-4911-9810-019fe28695e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:01.183387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb467d5-d32a-4911-9810-019fe28695e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:01.189002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568a7a1-13ff-4e5a-904b-4391481c0a67 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:01.198307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:01.199828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:01.202296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:01.202977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:01.203810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb467d5-d32a-4911-9810-019fe28695e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:01.205684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb467d5-d32a-4911-9810-019fe28695e7 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:01.223126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:01.232995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:01.237574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:01.256385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 858/1704] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:01.270571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 847/1705] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:02.217235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d896f5-5774-40c7-88ff-974c839a88d0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:02.218000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d896f5-5774-40c7-88ff-974c839a88d0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:02.218000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d896f5-5774-40c7-88ff-974c839a88d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:02.291277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-28d896f5-5774-40c7-88ff-974c839a88d0 None None] MFA Rules not processed for user `be34f77da4d8407e9a4333bc71f64595`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:02.292884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-28d896f5-5774-40c7-88ff-974c839a88d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:02.478047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 859/1706] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:02.485461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:02.487169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:02.747150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc9f3d-59e5-451a-b128-23e954c5722a service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:02.770599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:02.771691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:02.777466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:02.778084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:02.809862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:02.813156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:02.861391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 848/1707] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:02.984744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:02.985034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:02.985313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:03.052113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] MFA Rules not processed for user `cd235744a7ee462eb13e383930de4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:03.054616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:03.075518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 860/1708] 10.222.0.36 () {58 vars in 1015 bytes} [Sat Aug 6 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:03.139462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:03.141469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:03.184681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674e35f-0668-40bc-a6e9-4be15207c2db service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:03.194969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3378b2-aee7-441b-89f3-5e8e8277658d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:03.203897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3378b2-aee7-441b-89f3-5e8e8277658d service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:03.210328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3378b2-aee7-441b-89f3-5e8e8277658d service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:03.210800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3378b2-aee7-441b-89f3-5e8e8277658d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:03.231639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3378b2-aee7-441b-89f3-5e8e8277658d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:03.233939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3378b2-aee7-441b-89f3-5e8e8277658d service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:03.257276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 849/1709] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:04.661725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:04.663816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:04.692269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9421a-77f4-42d8-baea-66b3bd05be19 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:04.701699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:04.702617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:04.703093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:04.703520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] PATH_INFO: `/v3/projects/4ec92e2a9a9e47519d4003039ea1a0bc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:04.709474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec92e2a9a9e47519d4003039ea1a0bc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:04.711701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:04.832246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec92e2a9a9e47519d4003039ea1a0bc', 'request_id': 'req-f9c2aa1c-aef3-48df-89b6-fb830b065def'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:04.852474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec92e2a9a9e47519d4003039ea1a0bc', 'request_id': 'req-f9c2aa1c-aef3-48df-89b6-fb830b065def'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:04.868589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec92e2a9a9e47519d4003039ea1a0bc', 'request_id': 'req-f9c2aa1c-aef3-48df-89b6-fb830b065def'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:04.895072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] The token cache is being invalidate because project 4ec92e2a9a9e47519d4003039ea1a0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:04.895362 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:04.898409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:04.910988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 861/1710] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:04 2022] DELETE /identity/v3/projects/4ec92e2a9a9e47519d4003039ea1a0bc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:04.916778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937881cf-fe26-4a5c-a2f8-c43756fc749e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:04.917420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937881cf-fe26-4a5c-a2f8-c43756fc749e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:04.917420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937881cf-fe26-4a5c-a2f8-c43756fc749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:04.968156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-937881cf-fe26-4a5c-a2f8-c43756fc749e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:04.970942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-937881cf-fe26-4a5c-a2f8-c43756fc749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:04.973134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:04.974716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:05.011596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:05.190353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2aa1c-aef3-48df-89b6-fb830b065def admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:05.200162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 850/1711] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:05.201935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ce836-4dad-4f28-9f3f-3a78630e5a85 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:05.202225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ce836-4dad-4f28-9f3f-3a78630e5a85 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:05.202501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ce836-4dad-4f28-9f3f-3a78630e5a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:05.209427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:05.210097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:05.210375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:05.210648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] PATH_INFO: `/v3/projects/0efbe1d6f7094a8e9ded71857a5819cf` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:05.219895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efbe1d6f7094a8e9ded71857a5819cf)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:05.221861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:05.264487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a61ce836-4dad-4f28-9f3f-3a78630e5a85 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:05.265827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a61ce836-4dad-4f28-9f3f-3a78630e5a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:05.295649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 851/1712] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:05.299548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ce836-4dad-4f28-9f3f-3a78630e5a85 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:05.312663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ce836-4dad-4f28-9f3f-3a78630e5a85 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:05.348969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efbe1d6f7094a8e9ded71857a5819cf', 'request_id': 'req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.363956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efbe1d6f7094a8e9ded71857a5819cf', 'request_id': 'req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.378156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efbe1d6f7094a8e9ded71857a5819cf', 'request_id': 'req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.406321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] The token cache is being invalidate because project 0efbe1d6f7094a8e9ded71857a5819cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:05.406612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.407677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.409290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 862/1713] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:04 2022] DELETE /identity/v3/projects/0efbe1d6f7094a8e9ded71857a5819cf => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:05.413286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:05.416458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:05.446860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:05.447534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:05.447813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:05.448083 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:05.449368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:05.449609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:05.451447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:05.455483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:05.471265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 852/1714] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:05.479643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:05.481469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:05.508167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba4427-50ec-4cbe-ac15-3330ed407e6b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:05.525302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:05.525959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:05.526559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:05.526846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:05.527602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:05.529569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:05.583575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ae25-9e35-4d1e-8ab7-c9d24dfde908 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:05.597683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 853/1715] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:05 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:05.601031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:05.608930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:05.610273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:05.613841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:05.615118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] PATH_INFO: `/v3/users/1cdbf6de97c14d1b85d4e076f64fa223` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:05.618555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cdbf6de97c14d1b85d4e076f64fa223)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:05.646185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:05.782365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7b18-6567-452c-a12e-cf33767de874 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:05.788736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cdbf6de97c14d1b85d4e076f64fa223'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.788736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cdbf6de97c14d1b85d4e076f64fa223'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.789450 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cdbf6de97c14d1b85d4e076f64fa223'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.791300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:05.799521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:05.800171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:05.800452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:05.800724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:05.806230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:05.810240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:05.829238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cdbf6de97c14d1b85d4e076f64fa223'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.850485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cdbf6de97c14d1b85d4e076f64fa223'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:05.880851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 854/1716] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:05 2022] POST /identity/v3/users => generated 282 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:05.881653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:05.887607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:05.891232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 863/1717] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:05 2022] DELETE /identity/v3/users/1cdbf6de97c14d1b85d4e076f64fa223 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:05.919354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f082e-3b20-4ab3-a522-b87eba4de71f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:06.065897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:06.066555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:06.066837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:06.067133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:06.071889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:06.075042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:06.108957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 855/1718] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:06.112582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:06.154694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16fb99-6da9-423a-89c2-1fdda6e799c4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:06.189901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf20f-09ef-4c75-86a5-a193986dfac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:06.191312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf20f-09ef-4c75-86a5-a193986dfac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:06.195441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf20f-09ef-4c75-86a5-a193986dfac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:06.198390 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf20f-09ef-4c75-86a5-a193986dfac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:06.199652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf20f-09ef-4c75-86a5-a193986dfac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:06.201459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf20f-09ef-4c75-86a5-a193986dfac9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:06.222098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:06.246630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 864/1719] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:06.250762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae3d31-8c6e-4c86-8f48-d547b341cd85 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:06.272196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:06.273130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:06.273516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:06.273932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] PATH_INFO: `/v3/projects/342de7ca16784c0ba851d1c67d67da7f/users/faa122040bdc4a48bec1a5e7b451e93c/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:06.333055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=342de7ca16784c0ba851d1c67d67da7f, user_id=faa122040bdc4a48bec1a5e7b451e93c, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:06.339026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:06.391549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 856/1720] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:06 2022] PUT /identity/v3/projects/342de7ca16784c0ba851d1c67d67da7f/users/faa122040bdc4a48bec1a5e7b451e93c/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:06.398237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6da4-78ac-4ea9-84b0-66cb4068467b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:06.399283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6da4-78ac-4ea9-84b0-66cb4068467b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:06.399771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6da4-78ac-4ea9-84b0-66cb4068467b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:06.454462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4a4c6da4-78ac-4ea9-84b0-66cb4068467b None None] MFA Rules not processed for user `faa122040bdc4a48bec1a5e7b451e93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:06.456215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4a4c6da4-78ac-4ea9-84b0-66cb4068467b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:06.566925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 865/1721] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:06.571706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:06.638993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:06.684438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcf83a-cc84-454d-a472-ffb902b8d44d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:06.691098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:06.692139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:06.692524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:06.692926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:06.693708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:06.696196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:06.841423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6da4-78ac-4ea9-84b0-66cb4068467b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:06.852871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 857/1722] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:06 2022] POST /identity/v3/projects => generated 365 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:06.865454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6da4-78ac-4ea9-84b0-66cb4068467b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:06.989300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:06.990113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:06.990597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:06.991017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:06.991825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:06.994145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:07.038814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 866/1723] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:06 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:07.043992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:07.065207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf536231-6d34-4236-8c7a-16ad00667b35 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:07.086708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ab611-6110-49c6-8d49-43e1254703d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:07.088308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ab611-6110-49c6-8d49-43e1254703d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:07.088782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ab611-6110-49c6-8d49-43e1254703d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:07.089343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ab611-6110-49c6-8d49-43e1254703d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:07.090801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ab611-6110-49c6-8d49-43e1254703d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:07.093679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ab611-6110-49c6-8d49-43e1254703d8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:07.117603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 858/1724] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:07.120069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:07.135780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f595d02-18fc-436a-9ad5-1efc785465c3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:07.147892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b69b1-c05e-431e-b661-5afe9d2bf1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:07.148457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b69b1-c05e-431e-b661-5afe9d2bf1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:07.149267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b69b1-c05e-431e-b661-5afe9d2bf1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:07.149546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b69b1-c05e-431e-b661-5afe9d2bf1ac admin admin] PATH_INFO: `/v3/projects/844223d32a8d4306aa737f1d7d17c69b/users/73d74877d252440bbc8e648d5a4db229/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:07.194879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b69b1-c05e-431e-b661-5afe9d2bf1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=844223d32a8d4306aa737f1d7d17c69b, user_id=73d74877d252440bbc8e648d5a4db229, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:07.198164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b69b1-c05e-431e-b661-5afe9d2bf1ac admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:07.219057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8323b9-6291-4d48-806a-3eaf89988289 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:07.219674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8323b9-6291-4d48-806a-3eaf89988289 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:07.220288 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8323b9-6291-4d48-806a-3eaf89988289 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:07.222723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 859/1725] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:30:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:30:07.229606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 867/1726] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:07 2022] PUT /identity/v3/projects/844223d32a8d4306aa737f1d7d17c69b/users/73d74877d252440bbc8e648d5a4db229/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:07.234638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8323b9-6291-4d48-806a-3eaf89988289 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:07.273212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70609384-c6f4-4cf0-9921-de75a1d8289e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:07.273514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70609384-c6f4-4cf0-9921-de75a1d8289e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:07.274345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70609384-c6f4-4cf0-9921-de75a1d8289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:07.325497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-70609384-c6f4-4cf0-9921-de75a1d8289e None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:07.327947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-70609384-c6f4-4cf0-9921-de75a1d8289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:07.347770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:07.430705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8323b9-6291-4d48-806a-3eaf89988289 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:07.442847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:07.443705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:07.444141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:07.444566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:07.446494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:07.448172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:07.471460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 860/1727] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:07.473766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:07.499280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cd4e4-d5e1-43c9-b2f5-a28a726d0f8d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:07.517648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:07.518414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:07.531260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:07.531666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] PATH_INFO: `/v3/projects/844223d32a8d4306aa737f1d7d17c69b/users/73d74877d252440bbc8e648d5a4db229/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:07.556245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844223d32a8d4306aa737f1d7d17c69b, user_id=73d74877d252440bbc8e648d5a4db229, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:07.559527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:07.577602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 868/1728] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:30:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:07.606750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 861/1729] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:07 2022] PUT /identity/v3/projects/844223d32a8d4306aa737f1d7d17c69b/users/73d74877d252440bbc8e648d5a4db229/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:07.608348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:07.609075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:07.622028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:07.712689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] MFA Rules not processed for user `73d74877d252440bbc8e648d5a4db229`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:07.715194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:07.756452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:07.759659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:08.001960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 869/1730] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:08.005057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:08.008551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:08.215400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb8444-c968-474c-a731-0ec0e37d2f76 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:08.217971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a816f3-e6e9-4ce3-be04-8b0ec64623c6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:08.239281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:08.239922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:08.240274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:08.240574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:08.241448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df98649-c2a0-438f-a4f7-e3a7490f9ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:08.251122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df98649-c2a0-438f-a4f7-e3a7490f9ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:08.251504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df98649-c2a0-438f-a4f7-e3a7490f9ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:08.251883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df98649-c2a0-438f-a4f7-e3a7490f9ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:08.252332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:08.255036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:08.269929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 862/1731] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:08.275532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df98649-c2a0-438f-a4f7-e3a7490f9ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:08.277175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df98649-c2a0-438f-a4f7-e3a7490f9ab4 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:08.289300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 870/1732] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:08.470703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:08.485031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510233-ffde-422f-8744-eaf62856b25b service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:08.494526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:08.495690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:08.498105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:08.499000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:08.534364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:08.581383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803658d-5783-4f99-a5e9-64c47e2b2f6d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:08.585437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803658d-5783-4f99-a5e9-64c47e2b2f6d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:08.586008 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803658d-5783-4f99-a5e9-64c47e2b2f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:08.668652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3803658d-5783-4f99-a5e9-64c47e2b2f6d None None] MFA Rules not processed for user `5378c85cd31c42dfbc39ce738a8b6dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:08.669898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3803658d-5783-4f99-a5e9-64c47e2b2f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:08.681692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:08.684446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:08.725961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 863/1733] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:08.745580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:08.951645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 871/1734] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:08.969271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:08.970596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:09.005598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3751f-ea91-4503-aa7c-fe38f73b6209 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:09.013013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:09.014659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:09.015255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:09.015614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:09.030176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:09.031915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:09.063893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 864/1735] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:09.247987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803658d-5783-4f99-a5e9-64c47e2b2f6d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:09.262724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803658d-5783-4f99-a5e9-64c47e2b2f6d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:09.270560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:09.271625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:09.273306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:09.274842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:09.281139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:09.442721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:09.444254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:09.470326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 872/1736] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:09.995421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:09.997847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:10.155109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5b167-789e-4c2a-9d0b-5097a9fb6426 service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:10.169970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:10.171155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:10.171616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:10.172050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] PATH_INFO: `/v3/projects/9c75694ba7ce4d5183311f6519a14cf4` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:10.175900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c75694ba7ce4d5183311f6519a14cf4)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:10.178182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:10.318517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c75694ba7ce4d5183311f6519a14cf4', 'request_id': 'req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.333938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c75694ba7ce4d5183311f6519a14cf4', 'request_id': 'req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.352781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c75694ba7ce4d5183311f6519a14cf4', 'request_id': 'req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.382074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] The token cache is being invalidate because project 9c75694ba7ce4d5183311f6519a14cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:10.382479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.391063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3d7c4145-585e-4cbb-a601-1dae09fdbc7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.391063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 865/1737] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:09 2022] DELETE /identity/v3/projects/9c75694ba7ce4d5183311f6519a14cf4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:10.391704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:10.419242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b9b0-2efd-4706-b27f-07755cb1bbcb service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:10.664048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:10.679224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:10.682685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:10.683266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] PATH_INFO: `/v3/users/8c2bfdd87f344309ba264a0aeabab3c8` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:10.684813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c2bfdd87f344309ba264a0aeabab3c8)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:10.718829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:10.885383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c2bfdd87f344309ba264a0aeabab3c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.887981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c2bfdd87f344309ba264a0aeabab3c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.891033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c2bfdd87f344309ba264a0aeabab3c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.921464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:10.960017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c2bfdd87f344309ba264a0aeabab3c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:10.981797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6c45c92c-6e95-4f98-8beb-5003028e2aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c2bfdd87f344309ba264a0aeabab3c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:11.009317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 873/1738] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:10 2022] DELETE /identity/v3/users/8c2bfdd87f344309ba264a0aeabab3c8 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:12.140321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:12.140321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:12.140321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:12.242996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] MFA Rules not processed for user `8a1e8506e8b34a8d9c59b4acdbf23bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:12.245513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:12.347055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:12.547711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:12.548517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 866/1739] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:12.548956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:12.549575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:12.603102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] MFA Rules not processed for user `8a1e8506e8b34a8d9c59b4acdbf23bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:12.614648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:12.654638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 874/1740] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:12.659280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:12.660895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:12.674360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:12.677625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:12.995707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfce82c-217e-4833-adaa-afb0b251e432 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:13.022962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:13.023978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:13.024383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:13.027652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] PATH_INFO: `/v3/projects/a2b14fbcc21b45988a37758de43d061e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:13.031069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d84d79-13a7-4e21-9f30-398fa43ee7d3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:13.033500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b14fbcc21b45988a37758de43d061e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:13.036200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:13.047918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a8ec3-2dae-4690-8463-7a0078a698ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:13.050179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8ec3-2dae-4690-8463-7a0078a698ff service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:13.050700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8ec3-2dae-4690-8463-7a0078a698ff service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:13.052204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8ec3-2dae-4690-8463-7a0078a698ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:13.076516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a8ec3-2dae-4690-8463-7a0078a698ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:13.078530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a8ec3-2dae-4690-8463-7a0078a698ff service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:13.119474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 875/1741] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:13.205217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b14fbcc21b45988a37758de43d061e', 'request_id': 'req-1d73090a-c839-4ebf-8f1e-edfa671c817b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:13.225522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b14fbcc21b45988a37758de43d061e', 'request_id': 'req-1d73090a-c839-4ebf-8f1e-edfa671c817b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:13.243537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b14fbcc21b45988a37758de43d061e', 'request_id': 'req-1d73090a-c839-4ebf-8f1e-edfa671c817b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:13.278340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] The token cache is being invalidate because project a2b14fbcc21b45988a37758de43d061e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:13.278790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:13.284072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d73090a-c839-4ebf-8f1e-edfa671c817b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:13.286999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 867/1742] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:12 2022] DELETE /identity/v3/projects/a2b14fbcc21b45988a37758de43d061e => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:13.294084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d3969-b67e-47f4-8568-4199458e67b2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:13.294770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d3969-b67e-47f4-8568-4199458e67b2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:13.295304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d3969-b67e-47f4-8568-4199458e67b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:13.368527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-dc9d3969-b67e-47f4-8568-4199458e67b2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:13.369948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dc9d3969-b67e-47f4-8568-4199458e67b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:13.531415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 876/1743] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:13.533282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca59c1-1423-4585-9410-092394225b86 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:13.533881 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca59c1-1423-4585-9410-092394225b86 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:13.534297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca59c1-1423-4585-9410-092394225b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:13.566463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-54ca59c1-1423-4585-9410-092394225b86 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:13.569684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-54ca59c1-1423-4585-9410-092394225b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:13.587161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 868/1744] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:13.591843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d3969-b67e-47f4-8568-4199458e67b2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:13.608929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d3969-b67e-47f4-8568-4199458e67b2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:13.619127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:13.619781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:13.620067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:13.620342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:13.621057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:13.621366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:13.623076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:13.657697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 877/1745] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:13.663040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca59c1-1423-4585-9410-092394225b86 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:13.680634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca59c1-1423-4585-9410-092394225b86 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:13.689555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:13.690314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:13.691528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:13.691861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:13.692636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:13.696285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:13.748667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 869/1746] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:13 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:13.750140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:13.768342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9cb2-8394-4cc7-b173-df5b31c4ebf8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:13.838906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:13.921382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:13.922557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:13.923340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:13.923964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:13.924874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:13.927211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:13.975006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 878/1747] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:13 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:13.976075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:13.991624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3312316-48c2-4906-9d2c-8581ebc953b9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:14.016504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:14.017160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.017443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.025787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.030919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:14.034918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:14.082991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 870/1748] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:14.084612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:14.098731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585b639-800b-4a55-9486-422087a3cabe admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:14.120322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:14.124116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.124737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.125450 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.127612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:14.131632 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:14.134771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:14.137329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:14.151041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 879/1749] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:14.152271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:14.165694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822901c8-41b7-4c0f-a5a0-f812285e498b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:14.168949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deea6d-213f-4bb0-8858-016a984a3b44 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:14.181240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:14.182077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ce484-689f-4b2a-a773-496706c9a82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:14.184933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce484-689f-4b2a-a773-496706c9a82c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.185262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.185711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.186548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] PATH_INFO: `/v3/users/faa122040bdc4a48bec1a5e7b451e93c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.186857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce484-689f-4b2a-a773-496706c9a82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.188022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce484-689f-4b2a-a773-496706c9a82c admin admin] PATH_INFO: `/v3/projects/4cf6f14c91aa423d98207b2e11bf3f8f/users/e6cc48ddb71e4515bfdafd96e9877599/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.189770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faa122040bdc4a48bec1a5e7b451e93c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:14.193719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:14.222222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:14.236648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ce484-689f-4b2a-a773-496706c9a82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf6f14c91aa423d98207b2e11bf3f8f, user_id=e6cc48ddb71e4515bfdafd96e9877599, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:14.248890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ce484-689f-4b2a-a773-496706c9a82c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:14.306392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 880/1750] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:14 2022] PUT /identity/v3/projects/4cf6f14c91aa423d98207b2e11bf3f8f/users/e6cc48ddb71e4515bfdafd96e9877599/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:14.308267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.310136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.310589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.353425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faa122040bdc4a48bec1a5e7b451e93c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:14.376074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faa122040bdc4a48bec1a5e7b451e93c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:14.376997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faa122040bdc4a48bec1a5e7b451e93c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:14.380040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faa122040bdc4a48bec1a5e7b451e93c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:14.399450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b97cbf64-0690-4dd6-9937-43257594de31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faa122040bdc4a48bec1a5e7b451e93c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:14.402286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] MFA Rules not processed for user `e6cc48ddb71e4515bfdafd96e9877599`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:14.404993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:14.480037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 871/1751] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:14 2022] DELETE /identity/v3/users/faa122040bdc4a48bec1a5e7b451e93c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:14.485400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.486109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.487628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.560670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:14.562632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:14.653295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 881/1752] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:14.657816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:14.659343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:14.730757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 872/1753] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:14.732653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:14.738554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:14.877370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534968b-d783-4aa8-aaf9-d528711f757f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:14.896064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c3b71-e79d-4ecf-98dd-3690010f6f99 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:14.898087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:14.899508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.899945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.900370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.901110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638892dc-bf33-4f94-b1ae-790cf7e09622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:14.901382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:14.901953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638892dc-bf33-4f94-b1ae-790cf7e09622 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.902345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638892dc-bf33-4f94-b1ae-790cf7e09622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.902736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638892dc-bf33-4f94-b1ae-790cf7e09622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.904229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:14.914441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638892dc-bf33-4f94-b1ae-790cf7e09622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:14.916318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638892dc-bf33-4f94-b1ae-790cf7e09622 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:14.972820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 873/1754] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:14.975412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288eed4-304c-4da6-8e59-3bfe4f16cadc None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.978964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288eed4-304c-4da6-8e59-3bfe4f16cadc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.981080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288eed4-304c-4da6-8e59-3bfe4f16cadc None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:14.986016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 874/1755] 10.222.0.17 () {54 vars in 912 bytes} [Sat Aug 6 14:30:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:30:14.993817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 882/1756] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:14 2022] POST /identity/v3/projects => generated 379 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:14.999184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:14.999411 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:14.999643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.005363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:15.053671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:15.095570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] MFA Rules not processed for user `cd235744a7ee462eb13e383930de4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:15.099178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:15.221647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:15.229641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e0c20-ba40-41b7-aefb-f053a54913bf admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:15.258492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:15.260259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:15.265269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:15.265269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.267293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:15.275072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:15.292491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 875/1757] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:15.318383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:15.319961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:15.342615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 883/1758] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:14 2022] POST /identity/v3/users => generated 296 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:15.347644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:15.347933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf204e53-a783-4ebe-be7f-e2421aa4ca23 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:15.363896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:15.364542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:15.364823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:15.365097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.391801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202a7a3-37c6-498d-aa0b-a5c8ab0d95d8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:15.393275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:15.394738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:15.429507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:15.431966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:15.431966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:15.432154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.433048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:15.434355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 876/1759] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:15.443662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:15.489650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 884/1760] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:15.494448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:15.518541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c8ee-61d5-4743-9f28-5004d4c5b8a5 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:15.535702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:15.536407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:15.536690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:15.536966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] PATH_INFO: `/v3/projects/628cb2fa0c5a4c4d8fa7a5be6df434a2/users/efd8f784a20a4e7ca640b4b80aa6c29d/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.565151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=628cb2fa0c5a4c4d8fa7a5be6df434a2, user_id=efd8f784a20a4e7ca640b4b80aa6c29d, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:15.567858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:15.600834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 877/1761] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:15 2022] PUT /identity/v3/projects/628cb2fa0c5a4c4d8fa7a5be6df434a2/users/efd8f784a20a4e7ca640b4b80aa6c29d/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:15.605480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:15.618734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f9ea3-d558-4443-874c-b02784afee5a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:15.777126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc69141-6354-43a1-b67a-0a91abc1bcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:15.777830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc69141-6354-43a1-b67a-0a91abc1bcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:15.778099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc69141-6354-43a1-b67a-0a91abc1bcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:15.778304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc69141-6354-43a1-b67a-0a91abc1bcc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.779062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc69141-6354-43a1-b67a-0a91abc1bcc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:15.780723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc69141-6354-43a1-b67a-0a91abc1bcc0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:15.797720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 885/1762] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:15.799312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:15.823231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3f223-3b73-4997-bfac-11c10fdf71f9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:15.831682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:15.833486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:15.833774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:15.836160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] PATH_INFO: `/v3/projects/628cb2fa0c5a4c4d8fa7a5be6df434a2/users/efd8f784a20a4e7ca640b4b80aa6c29d/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.842923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=628cb2fa0c5a4c4d8fa7a5be6df434a2, user_id=efd8f784a20a4e7ca640b4b80aa6c29d, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:15.846155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:15.884239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 878/1763] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:15 2022] PUT /identity/v3/projects/628cb2fa0c5a4c4d8fa7a5be6df434a2/users/efd8f784a20a4e7ca640b4b80aa6c29d/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:15.887263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:15.887552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:15.887887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:15.954637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] MFA Rules not processed for user `efd8f784a20a4e7ca640b4b80aa6c29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:15.963002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:15.965563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:15.967361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:16.194908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 886/1764] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:16.203439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:16.206069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:16.209946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488bf0-9505-490a-b10d-1aa9ce52eab4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:16.222626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:16.223268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:16.223548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:16.224941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] PATH_INFO: `/v3/projects/342de7ca16784c0ba851d1c67d67da7f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:16.232226 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=342de7ca16784c0ba851d1c67d67da7f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:16.235500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:16.236218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:16.354346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '342de7ca16784c0ba851d1c67d67da7f', 'request_id': 'req-5095d1b3-5a04-4547-a30a-8683aa41c775'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.359292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9afb7-029d-42ca-a34a-f18d1c81447c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:16.379606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '342de7ca16784c0ba851d1c67d67da7f', 'request_id': 'req-5095d1b3-5a04-4547-a30a-8683aa41c775'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.399219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '342de7ca16784c0ba851d1c67d67da7f', 'request_id': 'req-5095d1b3-5a04-4547-a30a-8683aa41c775'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.426323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] The token cache is being invalidate because project 342de7ca16784c0ba851d1c67d67da7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:16.427677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.429453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.441015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 879/1765] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:15 2022] DELETE /identity/v3/projects/342de7ca16784c0ba851d1c67d67da7f => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:16.445516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:16.448053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:16.498791 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de34ca5-0f0f-4090-8f66-0e51b45776fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:16.499535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de34ca5-0f0f-4090-8f66-0e51b45776fb service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:16.499824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de34ca5-0f0f-4090-8f66-0e51b45776fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:16.500107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de34ca5-0f0f-4090-8f66-0e51b45776fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:16.506824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6de34ca5-0f0f-4090-8f66-0e51b45776fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:16.508965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:16.644290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095d1b3-5a04-4547-a30a-8683aa41c775 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:16.658672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:16.659543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:16.659930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:16.661250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] PATH_INFO: `/v3/users/73d74877d252440bbc8e648d5a4db229` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:16.665398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d74877d252440bbc8e648d5a4db229)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:16.680789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:16.690553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de34ca5-0f0f-4090-8f66-0e51b45776fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:16.692780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de34ca5-0f0f-4090-8f66-0e51b45776fb service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:16.723648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 887/1766] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:16.728841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4595-e9ba-4486-9f56-3b0983a8ea62 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:16.729124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4595-e9ba-4486-9f56-3b0983a8ea62 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:16.729568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4595-e9ba-4486-9f56-3b0983a8ea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:16.763861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0dbb4595-e9ba-4486-9f56-3b0983a8ea62 None None] MFA Rules not processed for user `be34f77da4d8407e9a4333bc71f64595`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:16.765764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0dbb4595-e9ba-4486-9f56-3b0983a8ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:16.913006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d74877d252440bbc8e648d5a4db229'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.977043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d74877d252440bbc8e648d5a4db229'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.980966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d74877d252440bbc8e648d5a4db229'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:16.987023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 888/1767] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:16.988055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d74877d252440bbc8e648d5a4db229'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:17.014387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d74877d252440bbc8e648d5a4db229'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:17.095050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 880/1768] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:16 2022] DELETE /identity/v3/users/73d74877d252440bbc8e648d5a4db229 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:17.559489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158af6-a828-4e3e-b480-5eedf9d937ff None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:17.559994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158af6-a828-4e3e-b480-5eedf9d937ff None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:17.560312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158af6-a828-4e3e-b480-5eedf9d937ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:17.649871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1c158af6-a828-4e3e-b480-5eedf9d937ff None None] MFA Rules not processed for user `a3e987dfd35f4a5da36d5034970fcf59`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:17.653590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1c158af6-a828-4e3e-b480-5eedf9d937ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:17.667376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:17.810710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 889/1769] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:18.114771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:18.116434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:18.637665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524a2a8-7887-4da0-b4e4-63d7e7b27823 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:18.655605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:18.657995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:18.659139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:18.661122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] PATH_INFO: `/v3/users/e6cc48ddb71e4515bfdafd96e9877599` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:18.664732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6cc48ddb71e4515bfdafd96e9877599)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:18.671296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:18.933192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:18.981732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6cc48ddb71e4515bfdafd96e9877599'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:19.008313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6cc48ddb71e4515bfdafd96e9877599'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:19.016478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6cc48ddb71e4515bfdafd96e9877599'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:19.019912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6cc48ddb71e4515bfdafd96e9877599'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:19.053390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6cc48ddb71e4515bfdafd96e9877599'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:19.118312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:19.119918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:19.120429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:19.129771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 881/1770] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:18 2022] DELETE /identity/v3/users/e6cc48ddb71e4515bfdafd96e9877599 => generated 0 bytes in 999 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:19.174739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:19.176289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:19.352378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 890/1771] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:19.373937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:19.375510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:19.580760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128dd06-4fcf-4609-bf54-abe722644a81 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:19.591819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d931b3-1d07-4fcf-8b93-01d760921566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:19.598447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d931b3-1d07-4fcf-8b93-01d760921566 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:19.598447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d931b3-1d07-4fcf-8b93-01d760921566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:19.598447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d931b3-1d07-4fcf-8b93-01d760921566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:19.635159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d931b3-1d07-4fcf-8b93-01d760921566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:19.636661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d931b3-1d07-4fcf-8b93-01d760921566 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:19.683329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:19.685839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:19.702586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 882/1772] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:20.058816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e572e-fe72-49d9-89db-85c41cdfccf4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:20.065821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:20.067329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:20.067674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:20.068052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:20.072451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:20.312162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:20.370574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a2615-7ace-458e-a836-d2850d777ac2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:20.371164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a2615-7ace-458e-a836-d2850d777ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:20.371248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a2615-7ace-458e-a836-d2850d777ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:20.405070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:20.406065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:20.424560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 891/1773] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:20.425507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:20.432134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:20.456993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9e9a2615-7ace-458e-a836-d2850d777ac2 None None] MFA Rules not processed for user `5378c85cd31c42dfbc39ce738a8b6dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:20.462390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9e9a2615-7ace-458e-a836-d2850d777ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:20.485503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26174c3d-83cf-497b-abac-edfca42d334c service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:20.498093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:20.498625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:20.499234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:20.499379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] PATH_INFO: `/v3/projects/844223d32a8d4306aa737f1d7d17c69b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:20.502760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=844223d32a8d4306aa737f1d7d17c69b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:20.504833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:20.620085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844223d32a8d4306aa737f1d7d17c69b', 'request_id': 'req-11ddfece-3413-4b86-9a50-6d21cfb894ed'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:20.628510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844223d32a8d4306aa737f1d7d17c69b', 'request_id': 'req-11ddfece-3413-4b86-9a50-6d21cfb894ed'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:20.639158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 883/1774] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:20.661012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844223d32a8d4306aa737f1d7d17c69b', 'request_id': 'req-11ddfece-3413-4b86-9a50-6d21cfb894ed'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:20.691538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] The token cache is being invalidate because project 844223d32a8d4306aa737f1d7d17c69b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:20.692174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:20.697003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-11ddfece-3413-4b86-9a50-6d21cfb894ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:20.706732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 892/1775] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:20 2022] DELETE /identity/v3/projects/844223d32a8d4306aa737f1d7d17c69b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:20.708443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b2ce1-0f35-43e0-b916-621fbc5fdec6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:20.708862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b2ce1-0f35-43e0-b916-621fbc5fdec6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:20.709161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b2ce1-0f35-43e0-b916-621fbc5fdec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:20.762402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-047b2ce1-0f35-43e0-b916-621fbc5fdec6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:20.763978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-047b2ce1-0f35-43e0-b916-621fbc5fdec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:20.890983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 884/1776] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:20.897191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080048d-dace-4505-b2aa-ccbc43b1bef0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:20.897694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080048d-dace-4505-b2aa-ccbc43b1bef0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:20.897971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080048d-dace-4505-b2aa-ccbc43b1bef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:20.947145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1080048d-dace-4505-b2aa-ccbc43b1bef0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:20.948786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1080048d-dace-4505-b2aa-ccbc43b1bef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:20.964392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 893/1777] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:20.965244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b2ce1-0f35-43e0-b916-621fbc5fdec6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:20.971991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:20.982843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b2ce1-0f35-43e0-b916-621fbc5fdec6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:20.997279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:20.997929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:20.998332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:20.998619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:20.999446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:20.999754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.001366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.038670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080048d-dace-4505-b2aa-ccbc43b1bef0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:21.052358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 885/1778] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:30:21.059624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080048d-dace-4505-b2aa-ccbc43b1bef0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:21.066786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:21.067948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.067948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.068660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.069699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.074137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.155502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 894/1779] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:21 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:21.160433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:21.167228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:21.170492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:21.204206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0b47-7ecc-412a-9b80-fe625cf7f240 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:21.357233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0495c5-0483-4c41-9a38-ada271f3f029 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:21.368002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:21.369429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.369970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.371663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] PATH_INFO: `/v3/projects/4cf6f14c91aa423d98207b2e11bf3f8f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.374352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf6f14c91aa423d98207b2e11bf3f8f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.377571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.389001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:21.389657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.391026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.393672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.395667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.398581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.458422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 886/1780] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:21 2022] POST /identity/v3/users => generated 291 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:21.461224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:21.474121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:21.479973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedddac-7761-4781-a2dc-d5e339349c37 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:21.493885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf6f14c91aa423d98207b2e11bf3f8f', 'request_id': 'req-6014dee0-ee39-4e47-8757-3cda495fd7f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.507072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf6f14c91aa423d98207b2e11bf3f8f', 'request_id': 'req-6014dee0-ee39-4e47-8757-3cda495fd7f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.526494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf6f14c91aa423d98207b2e11bf3f8f', 'request_id': 'req-6014dee0-ee39-4e47-8757-3cda495fd7f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.558815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] The token cache is being invalidate because project 4cf6f14c91aa423d98207b2e11bf3f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:21.559266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.562152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.567802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 895/1781] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:21 2022] DELETE /identity/v3/projects/4cf6f14c91aa423d98207b2e11bf3f8f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:21.571224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:21.574204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:21.627454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:21.628524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.629084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.629648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.630555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.632747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.655144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 887/1782] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:21.659249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:21.664339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:21.703420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87b51f-e2f6-48bb-8233-fe14dab04833 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:21.715302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:21.718046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.718437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.719154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.720140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.722028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.747087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 888/1783] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:21.748785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:21.766136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61a64-1a09-4c61-8f60-4a3b44897317 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:21.790241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014dee0-ee39-4e47-8757-3cda495fd7f7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:21.794417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851884-2943-4d16-8567-5d2beb47fbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:21.796720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851884-2943-4d16-8567-5d2beb47fbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.797720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851884-2943-4d16-8567-5d2beb47fbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.798501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851884-2943-4d16-8567-5d2beb47fbe2 admin admin] PATH_INFO: `/v3/projects/ba06335f582a4b3faf1320d7ed4e24f5/users/3fe21731f6834e60b42b74328c55f64e/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.800699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:21.801769 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.802165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.802453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] PATH_INFO: `/v3/users/efd8f784a20a4e7ca640b4b80aa6c29d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.804192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd8f784a20a4e7ca640b4b80aa6c29d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.806835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.848607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851884-2943-4d16-8567-5d2beb47fbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba06335f582a4b3faf1320d7ed4e24f5, user_id=3fe21731f6834e60b42b74328c55f64e, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:21.854605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851884-2943-4d16-8567-5d2beb47fbe2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:21.882244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 889/1784] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:21 2022] PUT /identity/v3/projects/ba06335f582a4b3faf1320d7ed4e24f5/users/3fe21731f6834e60b42b74328c55f64e/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:21.893623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174beb5-075a-400b-849f-1c425c748107 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:21.894043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174beb5-075a-400b-849f-1c425c748107 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:21.894444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174beb5-075a-400b-849f-1c425c748107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:21.905653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd8f784a20a4e7ca640b4b80aa6c29d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.908545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd8f784a20a4e7ca640b4b80aa6c29d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.909458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd8f784a20a4e7ca640b4b80aa6c29d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.945149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd8f784a20a4e7ca640b4b80aa6c29d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:21.967937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8174beb5-075a-400b-849f-1c425c748107 None None] MFA Rules not processed for user `3fe21731f6834e60b42b74328c55f64e`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:21.969329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8174beb5-075a-400b-849f-1c425c748107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:21.971980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd8f784a20a4e7ca640b4b80aa6c29d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:22.015320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 896/1785] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:21 2022] DELETE /identity/v3/users/efd8f784a20a4e7ca640b4b80aa6c29d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:22.053917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:22.153945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 890/1786] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:22.158305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:22.161380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:22.279948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:22.318754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78105cc1-34e3-4e3b-9dd9-501c0e9050f0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:22.324776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:22.325454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:22.325803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:22.326066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:22.326774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:22.329708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:22.380472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174beb5-075a-400b-849f-1c425c748107 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:22.381733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 897/1787] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:22 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:22.499228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174beb5-075a-400b-849f-1c425c748107 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:22.581999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:22.583095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:22.583575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:22.584067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:22.584826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:22.587354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:22.625305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 891/1788] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:22 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:22.628176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:22.665000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b9d65-9266-4d80-957f-f3dd4199448e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:22.680858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:22.681586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:22.681782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:22.682064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:22.682772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:22.684434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:22.726656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:22.728748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 898/1789] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:22.747992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee78239-497a-498f-8a58-7f7dccfcac0b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:22.765209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:22.766061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:22.766346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:22.766645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] PATH_INFO: `/v3/projects/d30ba605dd5b455690eeb360bdd748c7/users/032191498b8a407e91a84281f3a5ed9c/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:22.796087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30ba605dd5b455690eeb360bdd748c7, user_id=032191498b8a407e91a84281f3a5ed9c, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:22.798947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:22.833436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 892/1790] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:22 2022] PUT /identity/v3/projects/d30ba605dd5b455690eeb360bdd748c7/users/032191498b8a407e91a84281f3a5ed9c/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:22.838019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:22.862707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3194a-3897-4b5b-aa6a-84ec7a8fb807 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:22.974283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:22.977930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:22.980235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:22.980924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:22.981215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:22.981496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:22.983451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:22.985647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.001302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4893e-7664-47c0-a8b3-ebf280718634 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.009952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.011153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.011584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.012042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] PATH_INFO: `/v3/projects/628cb2fa0c5a4c4d8fa7a5be6df434a2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.013727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=628cb2fa0c5a4c4d8fa7a5be6df434a2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.015845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.019389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 899/1791] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:23.025457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.059494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570878e-9407-4f45-b327-2bdefaa623ea admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.064994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38fdb5-8afa-4dff-806e-f29629597a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.065769 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fdb5-8afa-4dff-806e-f29629597a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.066268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fdb5-8afa-4dff-806e-f29629597a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.067217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fdb5-8afa-4dff-806e-f29629597a32 admin admin] PATH_INFO: `/v3/projects/d30ba605dd5b455690eeb360bdd748c7/users/032191498b8a407e91a84281f3a5ed9c/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.069639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38fdb5-8afa-4dff-806e-f29629597a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30ba605dd5b455690eeb360bdd748c7, user_id=032191498b8a407e91a84281f3a5ed9c, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.072541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38fdb5-8afa-4dff-806e-f29629597a32 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.084244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:23.086942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 900/1792] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:23 2022] PUT /identity/v3/projects/d30ba605dd5b455690eeb360bdd748c7/users/032191498b8a407e91a84281f3a5ed9c/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:23.096521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.096700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628cb2fa0c5a4c4d8fa7a5be6df434a2', 'request_id': 'req-5e10f057-8464-4877-a7ae-fff629e20ba5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:23.097217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.097900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.104519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628cb2fa0c5a4c4d8fa7a5be6df434a2', 'request_id': 'req-5e10f057-8464-4877-a7ae-fff629e20ba5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:23.118855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628cb2fa0c5a4c4d8fa7a5be6df434a2', 'request_id': 'req-5e10f057-8464-4877-a7ae-fff629e20ba5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:23.141349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] MFA Rules not processed for user `032191498b8a407e91a84281f3a5ed9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:23.142607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:23.143531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] The token cache is being invalidate because project 628cb2fa0c5a4c4d8fa7a5be6df434a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:23.143826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:23.144561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5e10f057-8464-4877-a7ae-fff629e20ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:23.149791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 893/1793] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:22 2022] DELETE /identity/v3/projects/628cb2fa0c5a4c4d8fa7a5be6df434a2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:23.155385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5a6bb-c400-492c-9d07-0a31c5697b8e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.156703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5a6bb-c400-492c-9d07-0a31c5697b8e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.157340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5a6bb-c400-492c-9d07-0a31c5697b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.184251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-faf5a6bb-c400-492c-9d07-0a31c5697b8e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:23.185975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-faf5a6bb-c400-492c-9d07-0a31c5697b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:23.233851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 901/1794] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:23.242666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.247666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:23.309298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 894/1795] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:23.312119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:23.317423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac67674-b171-4cd6-8196-5e5a789fd737 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.317955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac67674-b171-4cd6-8196-5e5a789fd737 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.318620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac67674-b171-4cd6-8196-5e5a789fd737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.369536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc1aed-3d09-492d-8a86-0a1213a7aa29 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.372625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7ac67674-b171-4cd6-8196-5e5a789fd737 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:23.373977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7ac67674-b171-4cd6-8196-5e5a789fd737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:23.374272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.376814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.377099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.378803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.389499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.391257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 895/1796] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:23.391987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.393248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac67674-b171-4cd6-8196-5e5a789fd737 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.403400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac67674-b171-4cd6-8196-5e5a789fd737 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.405826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 902/1797] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:23.408122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.408754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.409071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.409353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.410093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:23.410398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.412110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.431546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 896/1798] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:23.436193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.457085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f749402-3ea8-41f1-9595-8251b64f4859 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.468010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.468998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.469332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.469817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.470720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.473402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.511511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 903/1799] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:23 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:23.515729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.527333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85be13e-77a0-4d1c-912b-e9a36d5001b8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.603567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.604375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.604653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.604920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.605616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.607699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.634903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 897/1800] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:23 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:23.638140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.650067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307261-af49-4f91-98d4-4724e01c03b0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.655806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.656622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.656914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.657196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.657958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.659916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.675219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 904/1801] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:23.678527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.691880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc35754-e20a-4a25-bc4b-543e76239b25 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.700768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e889ba-6e7c-47a5-a351-7a5d4032df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.701386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e889ba-6e7c-47a5-a351-7a5d4032df05 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.701775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e889ba-6e7c-47a5-a351-7a5d4032df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.702054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e889ba-6e7c-47a5-a351-7a5d4032df05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.705093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e889ba-6e7c-47a5-a351-7a5d4032df05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.714565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e889ba-6e7c-47a5-a351-7a5d4032df05 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.744208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 898/1802] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:23.748202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:23.761700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8a3f7-546a-4cd0-8963-573d7e7811a9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:23.774625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16309ac-d7c4-46c2-be8e-8ab4b93c8781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:23.776519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16309ac-d7c4-46c2-be8e-8ab4b93c8781 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.784801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16309ac-d7c4-46c2-be8e-8ab4b93c8781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.785133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16309ac-d7c4-46c2-be8e-8ab4b93c8781 admin admin] PATH_INFO: `/v3/projects/b48c7fb049374d908eaa587875479d8c/users/d7bd6f6a0194498db0005092eac4822f/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.797121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bccdc1-0c7f-4729-9c42-03b5cbe24ca1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.797392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bccdc1-0c7f-4729-9c42-03b5cbe24ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.797671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bccdc1-0c7f-4729-9c42-03b5cbe24ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.826921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16309ac-d7c4-46c2-be8e-8ab4b93c8781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48c7fb049374d908eaa587875479d8c, user_id=d7bd6f6a0194498db0005092eac4822f, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:23.829812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16309ac-d7c4-46c2-be8e-8ab4b93c8781 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:23.849712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-63bccdc1-0c7f-4729-9c42-03b5cbe24ca1 None None] MFA Rules not processed for user `a3e987dfd35f4a5da36d5034970fcf59`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:23.851076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-63bccdc1-0c7f-4729-9c42-03b5cbe24ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:23.873168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 905/1803] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:23 2022] PUT /identity/v3/projects/b48c7fb049374d908eaa587875479d8c/users/d7bd6f6a0194498db0005092eac4822f/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:23.874920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:23.882249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:23.883344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:23.891984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:23.958284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] MFA Rules not processed for user `d7bd6f6a0194498db0005092eac4822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:23.959255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 899/1804] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:23.959887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:23.974522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bccdc1-0c7f-4729-9c42-03b5cbe24ca1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.097368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bccdc1-0c7f-4729-9c42-03b5cbe24ca1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.104144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 906/1805] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:24.108677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '032191498b8a407e91a84281f3a5ed9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd30ba605dd5b455690eeb360bdd748c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.109326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.109610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.109886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.110597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:24.110922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:24.112713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:24.130777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 900/1806] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:24.131012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.132354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:24.135775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.161109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1e669-0845-4d9a-96bd-93e3430aa79e tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.176665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '032191498b8a407e91a84281f3a5ed9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd30ba605dd5b455690eeb360bdd748c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.177304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.177584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.177888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.178505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:24.181356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:24.231568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 901/1807] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:24 2022] POST /identity/v3/projects => generated 354 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:24.233049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.241396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4c2f5-ad2b-47c9-b1a1-c89c8c924306 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.262297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:24.297370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.298106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.298495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.298895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.313643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93e06c-3fb3-4fde-92ec-056e137ffada tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.321931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '032191498b8a407e91a84281f3a5ed9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd30ba605dd5b455690eeb360bdd748c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.322702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.323063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.323342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.324046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:24.324350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:24.326141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:24.343065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 902/1808] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:24.365095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:24.366527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:24.378837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 907/1809] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:24.387567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.399570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78eb3-298d-410f-9f36-1fb6479fe412 tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.405962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.407506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '032191498b8a407e91a84281f3a5ed9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd30ba605dd5b455690eeb360bdd748c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.408417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.408805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.409207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] PATH_INFO: `/v3/projects/4cabbf6ab27c4549bde081a6da71e16a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.410465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4cabbf6ab27c4549bde081a6da71e16a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:24.412727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:24.490896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:24.515454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cabbf6ab27c4549bde081a6da71e16a', 'request_id': 'req-0a09549c-13be-4690-965c-a96ba1a7fa8b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.522342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cabbf6ab27c4549bde081a6da71e16a', 'request_id': 'req-0a09549c-13be-4690-965c-a96ba1a7fa8b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.535871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cabbf6ab27c4549bde081a6da71e16a', 'request_id': 'req-0a09549c-13be-4690-965c-a96ba1a7fa8b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.552465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] The token cache is being invalidate because project 4cabbf6ab27c4549bde081a6da71e16a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:24.552764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.553494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a09549c-13be-4690-965c-a96ba1a7fa8b tempest-VolumeQuotaClassesTest-1874556403 tempest-VolumeQuotaClassesTest-1874556403-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.555347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 903/1810] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:24 2022] DELETE /identity/v3/projects/4cabbf6ab27c4549bde081a6da71e16a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:24.566004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a432e9b-2a11-4f15-8cfd-263270ec278b service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.642621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.643724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.644081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.644435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] PATH_INFO: `/v3/users/d7bd6f6a0194498db0005092eac4822f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.645741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7bd6f6a0194498db0005092eac4822f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:24.649317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:24.685674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:24.709489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7bd6f6a0194498db0005092eac4822f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.710182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7bd6f6a0194498db0005092eac4822f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.711467 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7bd6f6a0194498db0005092eac4822f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.727692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7bd6f6a0194498db0005092eac4822f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.739754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7bd6f6a0194498db0005092eac4822f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:24.765526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 908/1811] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:24 2022] DELETE /identity/v3/users/d7bd6f6a0194498db0005092eac4822f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:24.766810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.767175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.767457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.792349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:24.793964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:24.821064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.822693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:24.895369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 904/1812] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:24.903176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:24.908100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:24.923009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b6092-ba3c-468e-a43b-5f781fef649d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.928646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.929834 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.930569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.931042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] PATH_INFO: `/v3/users/3fe21731f6834e60b42b74328c55f64e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:24.932492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe21731f6834e60b42b74328c55f64e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:24.935231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:24.981133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1652-3a86-44af-9ba8-6a9c8f88b21f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:24.991614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f08be-7f58-42db-9e6e-404d5be1eea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:24.993372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f08be-7f58-42db-9e6e-404d5be1eea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:24.993758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f08be-7f58-42db-9e6e-404d5be1eea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:24.994164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f08be-7f58-42db-9e6e-404d5be1eea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:25.005696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f08be-7f58-42db-9e6e-404d5be1eea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:25.009208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f08be-7f58-42db-9e6e-404d5be1eea9 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:25.024729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 905/1813] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:25.024982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe21731f6834e60b42b74328c55f64e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.025847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe21731f6834e60b42b74328c55f64e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.027178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe21731f6834e60b42b74328c55f64e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.066753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe21731f6834e60b42b74328c55f64e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.080988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe21731f6834e60b42b74328c55f64e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.100532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 909/1814] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:24 2022] DELETE /identity/v3/users/3fe21731f6834e60b42b74328c55f64e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:25.104354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:25.104631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:25.104907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:25.128093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:25.129735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:25.215531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 906/1815] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:25.233170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:25.239560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:25.304142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:25.363508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82d96d-e4fa-4f15-811e-14cb6d14ebfc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:25.363508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f07616-779b-4178-93c2-5948fd05b95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:25.363508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f07616-779b-4178-93c2-5948fd05b95e service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:25.363508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f07616-779b-4178-93c2-5948fd05b95e service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:25.363882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f07616-779b-4178-93c2-5948fd05b95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:25.370060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f07616-779b-4178-93c2-5948fd05b95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:25.371538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f07616-779b-4178-93c2-5948fd05b95e service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:25.381414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 910/1816] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:25.621757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:25.624470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:25.638493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:25.644087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25bfb2-40ed-47fa-95a9-0afa6e3d749e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:25.652153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:25.653204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:25.653680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:25.654102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] PATH_INFO: `/v3/projects/b48c7fb049374d908eaa587875479d8c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:25.656188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48c7fb049374d908eaa587875479d8c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:25.659487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:25.800041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48c7fb049374d908eaa587875479d8c', 'request_id': 'req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.828493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48c7fb049374d908eaa587875479d8c', 'request_id': 'req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.853789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48c7fb049374d908eaa587875479d8c', 'request_id': 'req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.893205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] The token cache is being invalidate because project b48c7fb049374d908eaa587875479d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:25.893497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.895053 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9ebbed7f-81d8-4ab8-bdfd-3f26f16bfc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:25.901786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 907/1817] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:25 2022] DELETE /identity/v3/projects/b48c7fb049374d908eaa587875479d8c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:25.908098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6e078-be62-41df-9c0e-3266c5d2d916 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:25.908949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6e078-be62-41df-9c0e-3266c5d2d916 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:25.909529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6e078-be62-41df-9c0e-3266c5d2d916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:25.968504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6cc6e078-be62-41df-9c0e-3266c5d2d916 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:25.970219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6cc6e078-be62-41df-9c0e-3266c5d2d916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:26.070488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 911/1818] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:26.079014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76484ed2-a74e-48bd-a3fd-2a7f789d01cf None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.079304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76484ed2-a74e-48bd-a3fd-2a7f789d01cf None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.079581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76484ed2-a74e-48bd-a3fd-2a7f789d01cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.167442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-76484ed2-a74e-48bd-a3fd-2a7f789d01cf None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:26.169885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-76484ed2-a74e-48bd-a3fd-2a7f789d01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:26.195554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 908/1819] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:26.200655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6e078-be62-41df-9c0e-3266c5d2d916 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.221328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6e078-be62-41df-9c0e-3266c5d2d916 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.228935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.229653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.230643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.231469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.232279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:26.232683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.237495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.271662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 912/1820] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:26.277571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76484ed2-a74e-48bd-a3fd-2a7f789d01cf None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.290417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76484ed2-a74e-48bd-a3fd-2a7f789d01cf None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.301823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.302468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.303874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.304171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.305820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.306649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.308436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:26.312733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.365013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01190fe3-ba9d-4c49-9618-e8f5787d94b1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.374471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 909/1821] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:26 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:26.380113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.503227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afae715-33e9-4489-9710-c8794c40c41d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.505668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.506331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.506609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.507409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] PATH_INFO: `/v3/projects/ba06335f582a4b3faf1320d7ed4e24f5` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.509241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.509356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba06335f582a4b3faf1320d7ed4e24f5)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.509880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.510159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.510428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.511135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.511511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.513033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.540528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 910/1822] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:26 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:26.542762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.567206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8701c2a-10cd-4a9d-b87c-c1d3a7ff0ebc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.572973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.579844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.581013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.581013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.583228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.585811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.616663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 911/1823] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:26.618314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.619791 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba06335f582a4b3faf1320d7ed4e24f5', 'request_id': 'req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.630687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b329a13-8240-4642-afa3-fa13edecb55f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.638034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba06335f582a4b3faf1320d7ed4e24f5', 'request_id': 'req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.640190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:26.650853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba06335f582a4b3faf1320d7ed4e24f5', 'request_id': 'req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.671207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] The token cache is being invalidate because project ba06335f582a4b3faf1320d7ed4e24f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:26.671726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.673517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.678321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 913/1824] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:26 2022] DELETE /identity/v3/projects/ba06335f582a4b3faf1320d7ed4e24f5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:26.678484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:26.681038 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.682858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:26.718272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.718933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.719217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.719505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.720252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.723108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.737634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 912/1825] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:26.741458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.744400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:26.757421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925adb5f-465b-43ce-8e82-7516a1f2680e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.761115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0dfe9-85b0-4dbc-a02f-668c0ccd1879 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:26.763102 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07fe44-5cf6-4d86-a70e-b127528bd82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.764041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07fe44-5cf6-4d86-a70e-b127528bd82f admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.764329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07fe44-5cf6-4d86-a70e-b127528bd82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.764613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07fe44-5cf6-4d86-a70e-b127528bd82f admin admin] PATH_INFO: `/v3/projects/614cf1cf0c52442090e8c33e477f7a81/users/dab22fbabda84ca5a1db84e4d02d9270/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.766725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:26.767810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.768303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.768736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] PATH_INFO: `/v3/users/032191498b8a407e91a84281f3a5ed9c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.770128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] RBAC: Authorizing `identity:delete_user(user_id=032191498b8a407e91a84281f3a5ed9c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.772160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.784254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07fe44-5cf6-4d86-a70e-b127528bd82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=614cf1cf0c52442090e8c33e477f7a81, user_id=dab22fbabda84ca5a1db84e4d02d9270, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:26.787703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07fe44-5cf6-4d86-a70e-b127528bd82f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:26.817319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 913/1826] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:26 2022] PUT /identity/v3/projects/614cf1cf0c52442090e8c33e477f7a81/users/dab22fbabda84ca5a1db84e4d02d9270/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:26.828822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:26.829140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:26.829423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:26.872500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032191498b8a407e91a84281f3a5ed9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.873103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032191498b8a407e91a84281f3a5ed9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.873823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032191498b8a407e91a84281f3a5ed9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.885060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] MFA Rules not processed for user `dab22fbabda84ca5a1db84e4d02d9270`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:26.886632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:26.887662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032191498b8a407e91a84281f3a5ed9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.896754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032191498b8a407e91a84281f3a5ed9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:26.920533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 914/1827] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:26 2022] DELETE /identity/v3/users/032191498b8a407e91a84281f3a5ed9c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:26.958674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 914/1828] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:26.969780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:26.971214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:27.035759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d476895-a012-4edc-942f-d468ebd2bbac admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.040798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0604ca-6efa-4e61-9ad2-2208ac63fa36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.041449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0604ca-6efa-4e61-9ad2-2208ac63fa36 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.041778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0604ca-6efa-4e61-9ad2-2208ac63fa36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.042001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0604ca-6efa-4e61-9ad2-2208ac63fa36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.058745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:27.117721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0604ca-6efa-4e61-9ad2-2208ac63fa36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.119225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0604ca-6efa-4e61-9ad2-2208ac63fa36 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.129578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 915/1829] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:27.229991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:27.231463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:27.316058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717147f-e161-492e-8d27-59edeb7ae91c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.369044 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:27.383875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.384599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.384884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.385160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] PATH_INFO: `/v3/projects/d30ba605dd5b455690eeb360bdd748c7` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.386376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30ba605dd5b455690eeb360bdd748c7)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.388969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.441287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30ba605dd5b455690eeb360bdd748c7', 'request_id': 'req-836738e7-72d7-46dd-bb80-1eef49dacc63'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:27.450701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30ba605dd5b455690eeb360bdd748c7', 'request_id': 'req-836738e7-72d7-46dd-bb80-1eef49dacc63'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:27.460181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30ba605dd5b455690eeb360bdd748c7', 'request_id': 'req-836738e7-72d7-46dd-bb80-1eef49dacc63'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:27.474277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] The token cache is being invalidate because project d30ba605dd5b455690eeb360bdd748c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:27.474573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:27.475266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-836738e7-72d7-46dd-bb80-1eef49dacc63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:27.476869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 915/1830] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:27 2022] DELETE /identity/v3/projects/d30ba605dd5b455690eeb360bdd748c7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:27.481726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622daef4-5a9f-41ec-ac0d-2700dd5531d5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.482021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622daef4-5a9f-41ec-ac0d-2700dd5531d5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.482306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622daef4-5a9f-41ec-ac0d-2700dd5531d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.513382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-622daef4-5a9f-41ec-ac0d-2700dd5531d5 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:27.514606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-622daef4-5a9f-41ec-ac0d-2700dd5531d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:27.577110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 916/1831] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:27.585009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340affe-1331-439b-889e-9def63503f40 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.585299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340affe-1331-439b-889e-9def63503f40 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.586239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340affe-1331-439b-889e-9def63503f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.605894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4340affe-1331-439b-889e-9def63503f40 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:27.607120 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4340affe-1331-439b-889e-9def63503f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:27.615281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 916/1832] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:27.619687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622daef4-5a9f-41ec-ac0d-2700dd5531d5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:27.629325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622daef4-5a9f-41ec-ac0d-2700dd5531d5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.634398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.635045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.635324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.635591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.636277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:27.636583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.638315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.650695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 917/1833] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:27.655079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340affe-1331-439b-889e-9def63503f40 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:27.666168 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340affe-1331-439b-889e-9def63503f40 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.670951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.671516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.671797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.672070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.672740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.674648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.704073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 917/1834] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:27 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:27.714406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:27.724473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de62eb-36cc-4c78-8d03-40f6a7dce83b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.779949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.780500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.780783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.781056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.781716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.783683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.805877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 918/1835] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:27 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:27.809938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:27.820930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187332-bbcd-4cc8-88ab-81b7d32766e9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.825438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.826044 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.826321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.826591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.827342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.828921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.838916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 918/1836] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:27.842689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:27.853316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ab065-7908-42bb-9304-a726b5e8060d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.858247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472e3bb-5585-4489-8e5e-59adc2a1e7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.858886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e3bb-5585-4489-8e5e-59adc2a1e7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.859193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e3bb-5585-4489-8e5e-59adc2a1e7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.859470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e3bb-5585-4489-8e5e-59adc2a1e7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.860194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472e3bb-5585-4489-8e5e-59adc2a1e7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.861776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472e3bb-5585-4489-8e5e-59adc2a1e7e8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.874579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 919/1837] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:27.879403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:27.891903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d848c-3058-47ee-89dd-58b6e9e2e82d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:27.896994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:27.897750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.898124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.898480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] PATH_INFO: `/v3/projects/23ebc024317a44f69108ef9618d99db5/users/c7d63729829b4df8b2ee1becbeacc99c/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.915316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ebc024317a44f69108ef9618d99db5, user_id=c7d63729829b4df8b2ee1becbeacc99c, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:27.918050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:27.936321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 919/1838] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:27 2022] PUT /identity/v3/projects/23ebc024317a44f69108ef9618d99db5/users/c7d63729829b4df8b2ee1becbeacc99c/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:27.948006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c4131-e53f-4efc-891a-6f9efa292f71 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:27.948517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c4131-e53f-4efc-891a-6f9efa292f71 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:27.948592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c4131-e53f-4efc-891a-6f9efa292f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:27.984690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a51c4131-e53f-4efc-891a-6f9efa292f71 None None] MFA Rules not processed for user `c7d63729829b4df8b2ee1becbeacc99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:27.986214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a51c4131-e53f-4efc-891a-6f9efa292f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:27.997660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:28.087249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 920/1839] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:28.096870 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.165444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ecbda-0175-44b5-b031-8f68608762b7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.170681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.171335 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.171616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.171894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.172596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.174509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.202505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:28.213091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 920/1840] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:28 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:28.217108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c4131-e53f-4efc-891a-6f9efa292f71 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.231639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c4131-e53f-4efc-891a-6f9efa292f71 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.293745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.294396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.294681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.294990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.295661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.297605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.326177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 921/1841] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:28 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:28.330133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.340542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dda23-b2f6-4356-b98d-1f1b6b275b67 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.345150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.345761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.346038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.346309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.347071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.348727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.363370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 921/1842] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:28.364655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.375565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927d47e-06d0-4758-b169-02ed25f67e16 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.380234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.380897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.381175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.381446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] PATH_INFO: `/v3/projects/834a04b989fd4b36ade55da4317659de/users/9b6be2b490b547eead35c65de6012d44/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.396104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=834a04b989fd4b36ade55da4317659de, user_id=9b6be2b490b547eead35c65de6012d44, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.398749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.412849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 922/1843] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:28 2022] PUT /identity/v3/projects/834a04b989fd4b36ade55da4317659de/users/9b6be2b490b547eead35c65de6012d44/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:28.417346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.427686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87a06-1c33-4a16-b4d6-182a70e0893e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.485326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b58c8-2915-4c18-968f-d52669a5a8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.485978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b58c8-2915-4c18-968f-d52669a5a8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.486257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b58c8-2915-4c18-968f-d52669a5a8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.486530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b58c8-2915-4c18-968f-d52669a5a8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.487246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b58c8-2915-4c18-968f-d52669a5a8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.488864 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b58c8-2915-4c18-968f-d52669a5a8e6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.499542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 922/1844] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:28.503591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.513858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85effec2-0b49-4235-8995-312dcc4adccc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.519363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.520105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.520387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.520660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] PATH_INFO: `/v3/projects/834a04b989fd4b36ade55da4317659de/users/9b6be2b490b547eead35c65de6012d44/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.523390 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834a04b989fd4b36ade55da4317659de, user_id=9b6be2b490b547eead35c65de6012d44, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.526103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.537510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 923/1845] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:28 2022] PUT /identity/v3/projects/834a04b989fd4b36ade55da4317659de/users/9b6be2b490b547eead35c65de6012d44/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:28.551379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d0b77-af71-410f-9e5e-43dbe0cf6856 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.551626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d0b77-af71-410f-9e5e-43dbe0cf6856 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.552051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d0b77-af71-410f-9e5e-43dbe0cf6856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.580871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-311d0b77-af71-410f-9e5e-43dbe0cf6856 None None] MFA Rules not processed for user `9b6be2b490b547eead35c65de6012d44`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:28.582095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-311d0b77-af71-410f-9e5e-43dbe0cf6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:28.660785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 923/1846] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:28.673495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.675229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:28.752759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94892b-8bbd-471c-a804-28427aa44834 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.756472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ea6d3-7eb2-416c-b50e-986795bb50c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.757141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea6d3-7eb2-416c-b50e-986795bb50c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.757591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea6d3-7eb2-416c-b50e-986795bb50c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.757866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea6d3-7eb2-416c-b50e-986795bb50c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.767623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ea6d3-7eb2-416c-b50e-986795bb50c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.769035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ea6d3-7eb2-416c-b50e-986795bb50c1 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.779699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 924/1847] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:28.869138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d0b77-af71-410f-9e5e-43dbe0cf6856 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:28.878558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d0b77-af71-410f-9e5e-43dbe0cf6856 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:28.882236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ab791-fdd4-4e4f-a7dd-b8ce3dbb67c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:28.882847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab791-fdd4-4e4f-a7dd-b8ce3dbb67c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:28.883165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab791-fdd4-4e4f-a7dd-b8ce3dbb67c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:28.883442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab791-fdd4-4e4f-a7dd-b8ce3dbb67c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:28.939252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab791-fdd4-4e4f-a7dd-b8ce3dbb67c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:28.940717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab791-fdd4-4e4f-a7dd-b8ce3dbb67c4 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:28.950253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 924/1848] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:38.775593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:38.946825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:39.347842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145fb34-3da4-4439-899d-ff3d81ad6385 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:39.348923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145fb34-3da4-4439-899d-ff3d81ad6385 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:39.349514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145fb34-3da4-4439-899d-ff3d81ad6385 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:39.351637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 925/1849] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:30:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:30:39.388519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02afac-356b-480c-b6c2-5aadce3a2d9f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:39.388519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02afac-356b-480c-b6c2-5aadce3a2d9f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:39.388519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02afac-356b-480c-b6c2-5aadce3a2d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:39.448797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7c02afac-356b-480c-b6c2-5aadce3a2d9f None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:39.460916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7c02afac-356b-480c-b6c2-5aadce3a2d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:39.710491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 925/1850] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:30:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:39.876383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145fb34-3da4-4439-899d-ff3d81ad6385 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:39.880867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8145fb34-3da4-4439-899d-ff3d81ad6385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:39.958847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145fb34-3da4-4439-899d-ff3d81ad6385 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:39.997405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dbbde-4219-4c34-8dad-318f3a697e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:39.998056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dbbde-4219-4c34-8dad-318f3a697e51 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:39.998341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dbbde-4219-4c34-8dad-318f3a697e51 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:39.998617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dbbde-4219-4c34-8dad-318f3a697e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:40.031841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dbbde-4219-4c34-8dad-318f3a697e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:40.033765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dbbde-4219-4c34-8dad-318f3a697e51 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:40.073568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 926/1851] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:42.094531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce11c9-c16a-4bd2-bd04-85ef48142412 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:42.095064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce11c9-c16a-4bd2-bd04-85ef48142412 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:42.095064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce11c9-c16a-4bd2-bd04-85ef48142412 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:42.100929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 926/1852] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:30:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:30:42.146463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919f98a-2320-4455-9d1a-e10ee831cd0a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:42.147738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919f98a-2320-4455-9d1a-e10ee831cd0a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:42.149175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919f98a-2320-4455-9d1a-e10ee831cd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:42.194626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9919f98a-2320-4455-9d1a-e10ee831cd0a None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:42.196418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9919f98a-2320-4455-9d1a-e10ee831cd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:42.211387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 927/1853] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:42.379403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce11c9-c16a-4bd2-bd04-85ef48142412 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:42.404049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce11c9-c16a-4bd2-bd04-85ef48142412 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:42.422212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3423d73-568a-4fb9-acfe-1fca34fab2fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:42.422851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3423d73-568a-4fb9-acfe-1fca34fab2fe service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:42.423182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3423d73-568a-4fb9-acfe-1fca34fab2fe service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:42.423461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3423d73-568a-4fb9-acfe-1fca34fab2fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:42.434345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3423d73-568a-4fb9-acfe-1fca34fab2fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:42.436463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3423d73-568a-4fb9-acfe-1fca34fab2fe service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:42.449818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 927/1854] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:46.374032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919f98a-2320-4455-9d1a-e10ee831cd0a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:46.459224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919f98a-2320-4455-9d1a-e10ee831cd0a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:46.466689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:46.467567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:46.467949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:46.468356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] PATH_INFO: `/v3/users/c7d63729829b4df8b2ee1becbeacc99c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:46.469901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d63729829b4df8b2ee1becbeacc99c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:46.471994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:46.536501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d63729829b4df8b2ee1becbeacc99c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:46.537340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d63729829b4df8b2ee1becbeacc99c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:46.538052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d63729829b4df8b2ee1becbeacc99c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:46.555651 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d63729829b4df8b2ee1becbeacc99c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:46.574321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d63729829b4df8b2ee1becbeacc99c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:46.592350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 928/1855] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:46 2022] DELETE /identity/v3/users/c7d63729829b4df8b2ee1becbeacc99c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:46.597378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:46.597867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:46.597962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:46.622181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:46.623677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:46.701626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 928/1856] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:46.712307 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:46.713732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:46.778538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db23ef-71a7-46af-958c-9b449204ceea admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:46.782283 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:46.783045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:46.783330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:46.783606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:46.795374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:46.796785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:46.807863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 929/1857] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:47.102888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:47.104738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:47.117350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909236-45f4-4db0-9b7c-622ba7e6d715 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:47.121982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:47.122707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:47.123094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:47.123444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] PATH_INFO: `/v3/projects/23ebc024317a44f69108ef9618d99db5` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:47.124777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ebc024317a44f69108ef9618d99db5)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:47.126862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:47.181279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ebc024317a44f69108ef9618d99db5', 'request_id': 'req-c95ca018-d770-4137-85d5-fddcbae2be07'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.188641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ebc024317a44f69108ef9618d99db5', 'request_id': 'req-c95ca018-d770-4137-85d5-fddcbae2be07'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.198110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ebc024317a44f69108ef9618d99db5', 'request_id': 'req-c95ca018-d770-4137-85d5-fddcbae2be07'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.209330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] The token cache is being invalidate because project 23ebc024317a44f69108ef9618d99db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:47.209626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.210274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.212448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 929/1858] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:47 2022] DELETE /identity/v3/projects/23ebc024317a44f69108ef9618d99db5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:47.217073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:47.222392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:47.229140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b671d40-ab0c-468a-be76-28ee3ccfcb40 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:47.288920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:47.289554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:47.289836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:47.290112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] PATH_INFO: `/v3/users/9b6be2b490b547eead35c65de6012d44` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:47.291321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6be2b490b547eead35c65de6012d44)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:47.293218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:47.321214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:47.322846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:47.331217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:47.349583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca018-d770-4137-85d5-fddcbae2be07 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:47.353136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6be2b490b547eead35c65de6012d44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.355312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6be2b490b547eead35c65de6012d44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.356089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6be2b490b547eead35c65de6012d44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.362936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:47.363767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:47.364181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:47.364599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] PATH_INFO: `/v3/users/dab22fbabda84ca5a1db84e4d02d9270` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:47.366042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab22fbabda84ca5a1db84e4d02d9270)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:47.370357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:47.375012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6be2b490b547eead35c65de6012d44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.385739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-93608da7-372a-4a96-bdd1-b7e7795ded1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6be2b490b547eead35c65de6012d44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.414934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 930/1859] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:47 2022] DELETE /identity/v3/users/9b6be2b490b547eead35c65de6012d44 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:47.454244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab22fbabda84ca5a1db84e4d02d9270'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.463068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab22fbabda84ca5a1db84e4d02d9270'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.464210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab22fbabda84ca5a1db84e4d02d9270'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.465043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab22fbabda84ca5a1db84e4d02d9270'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.476465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab22fbabda84ca5a1db84e4d02d9270'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:47.518291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 930/1860] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:47 2022] DELETE /identity/v3/users/dab22fbabda84ca5a1db84e4d02d9270 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:47.523445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a190c9-743b-4a7a-868d-848656844680 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:47.523957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a190c9-743b-4a7a-868d-848656844680 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:47.524461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a190c9-743b-4a7a-868d-848656844680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:47.551796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-38a190c9-743b-4a7a-868d-848656844680 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:47.553844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-38a190c9-743b-4a7a-868d-848656844680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:47.692403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 931/1861] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:47.722327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:47.723943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:47.820089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fc8e9-9015-411a-aa17-0f65638ddff3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:47.824620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d548a1-efc4-4f8e-a335-04a9cbf03630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:47.825391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d548a1-efc4-4f8e-a335-04a9cbf03630 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:47.825772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d548a1-efc4-4f8e-a335-04a9cbf03630 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:47.826147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d548a1-efc4-4f8e-a335-04a9cbf03630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:47.857046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d548a1-efc4-4f8e-a335-04a9cbf03630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:47.859135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d548a1-efc4-4f8e-a335-04a9cbf03630 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:47.872080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 931/1862] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:47.970832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a190c9-743b-4a7a-868d-848656844680 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:47.972616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-38a190c9-743b-4a7a-868d-848656844680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:47.996280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a190c9-743b-4a7a-868d-848656844680 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.006783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.007942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.008313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.008985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] PATH_INFO: `/v3/projects/834a04b989fd4b36ade55da4317659de` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.010489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=834a04b989fd4b36ade55da4317659de)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.012934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.032482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:48.074202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834a04b989fd4b36ade55da4317659de', 'request_id': 'req-df8c2ecc-6146-4abc-bba2-e98917155f4b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.080615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834a04b989fd4b36ade55da4317659de', 'request_id': 'req-df8c2ecc-6146-4abc-bba2-e98917155f4b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.086929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834a04b989fd4b36ade55da4317659de', 'request_id': 'req-df8c2ecc-6146-4abc-bba2-e98917155f4b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.102584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] The token cache is being invalidate because project 834a04b989fd4b36ade55da4317659de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:48.102860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.103537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.105105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 932/1863] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:47 2022] DELETE /identity/v3/projects/834a04b989fd4b36ade55da4317659de => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:48.110306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a2bd-a8da-4da2-a5c5-13147695b6c4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.110614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a2bd-a8da-4da2-a5c5-13147695b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.110943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a2bd-a8da-4da2-a5c5-13147695b6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.131285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2228a2bd-a8da-4da2-a5c5-13147695b6c4 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:48.132555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2228a2bd-a8da-4da2-a5c5-13147695b6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:48.190435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.191863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:48.195870 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 932/1864] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:48.205304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf6aaf-0029-4b5e-bea8-b45ff51784fa None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.205603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf6aaf-0029-4b5e-bea8-b45ff51784fa None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.205918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf6aaf-0029-4b5e-bea8-b45ff51784fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.209024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c2ecc-6146-4abc-bba2-e98917155f4b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.214275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.215045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.215383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.215662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] PATH_INFO: `/v3/projects/614cf1cf0c52442090e8c33e477f7a81` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.219220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614cf1cf0c52442090e8c33e477f7a81)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.221263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.226017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b1cf6aaf-0029-4b5e-bea8-b45ff51784fa None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:48.227792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b1cf6aaf-0029-4b5e-bea8-b45ff51784fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:48.236300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 933/1865] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:48.240625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf6aaf-0029-4b5e-bea8-b45ff51784fa None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.250065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf6aaf-0029-4b5e-bea8-b45ff51784fa None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.254965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.255567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.255844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.256114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.256802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:48.257095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.258697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.265525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:48.269678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614cf1cf0c52442090e8c33e477f7a81', 'request_id': 'req-f0731137-2d1b-4a83-a069-148103b19e56'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.271178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 934/1866] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:30:48.275510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.277470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614cf1cf0c52442090e8c33e477f7a81', 'request_id': 'req-f0731137-2d1b-4a83-a069-148103b19e56'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.284021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614cf1cf0c52442090e8c33e477f7a81', 'request_id': 'req-f0731137-2d1b-4a83-a069-148103b19e56'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.285569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbb9e0-ac3c-4cc0-8337-5247155dca3d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.297211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] The token cache is being invalidate because project 614cf1cf0c52442090e8c33e477f7a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:48.297502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.298137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f0731137-2d1b-4a83-a069-148103b19e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:48.299942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 933/1867] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:48 2022] DELETE /identity/v3/projects/614cf1cf0c52442090e8c33e477f7a81 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:48.304410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd874ab4-3d23-49a5-ab35-7fb6a34ac098 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.304691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd874ab4-3d23-49a5-ab35-7fb6a34ac098 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.304965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd874ab4-3d23-49a5-ab35-7fb6a34ac098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.324251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fd874ab4-3d23-49a5-ab35-7fb6a34ac098 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:48.325443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fd874ab4-3d23-49a5-ab35-7fb6a34ac098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:48.343417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.344048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.344329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.344607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.345271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.347289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.378633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 935/1868] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:48.379619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.381251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:48.386078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 934/1869] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:48.393407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405e08-6bd7-4f3f-af6a-0f1c85817555 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.393769 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405e08-6bd7-4f3f-af6a-0f1c85817555 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.394052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405e08-6bd7-4f3f-af6a-0f1c85817555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.414306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6d405e08-6bd7-4f3f-af6a-0f1c85817555 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:48.415622 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6d405e08-6bd7-4f3f-af6a-0f1c85817555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:48.440019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:48.445810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26cb81-6e2a-4558-a237-fa281b491aa0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.451229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.451856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.452188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.452413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.453089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.455199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.476533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 936/1870] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:48.479253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 935/1871] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:48.480772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.483153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d405e08-6bd7-4f3f-af6a-0f1c85817555 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.490333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb628ce2-6f5e-442d-94db-53defbc810b7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.492886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d405e08-6bd7-4f3f-af6a-0f1c85817555 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.495026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.495623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.495900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.496174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.496854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.497642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.498243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.498524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.498621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.498802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.499527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:48.499838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.501407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.509079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 937/1872] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:48.512962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.513368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 936/1873] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:30:48.517440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.525198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7388-49cb-4e9d-9313-8e76a4173efb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.527657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edf27-a5fe-4287-a0bb-16c122bb03a2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.529784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.530379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.530652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.530957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.531641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.532338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.532927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.533212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.533313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.533489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.534146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.536057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.543474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 938/1874] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:48.547355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.560573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03de9a8-ea69-4289-8efa-b992712710e2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.566135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f030e-392a-4e67-8ea8-975a734c0c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.566822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f030e-392a-4e67-8ea8-975a734c0c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.567076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f030e-392a-4e67-8ea8-975a734c0c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.567359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f030e-392a-4e67-8ea8-975a734c0c36 admin admin] PATH_INFO: `/v3/projects/e070800e91cc46deaa9344ac32d6353a/users/3ac6ca1bb08b43228cbae335fac123a6/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.572853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 937/1875] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:48.573972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.586415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f030e-392a-4e67-8ea8-975a734c0c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e070800e91cc46deaa9344ac32d6353a, user_id=3ac6ca1bb08b43228cbae335fac123a6, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.589323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f030e-392a-4e67-8ea8-975a734c0c36 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.605958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 939/1876] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:48 2022] PUT /identity/v3/projects/e070800e91cc46deaa9344ac32d6353a/users/3ac6ca1bb08b43228cbae335fac123a6/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:48.616555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b384-908e-478f-8873-015ba944f5e1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.616908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b384-908e-478f-8873-015ba944f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.617330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b384-908e-478f-8873-015ba944f5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.639206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b3727-f783-418d-8515-b5fdb149b21f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.649834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0991b384-908e-478f-8873-015ba944f5e1 None None] MFA Rules not processed for user `3ac6ca1bb08b43228cbae335fac123a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:48.651800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0991b384-908e-478f-8873-015ba944f5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:48.665760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:48.704811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.705545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.705901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.706251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.707290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.709191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.755529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 940/1877] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:48.755873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 938/1878] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:48.756056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991b384-908e-478f-8873-015ba944f5e1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.756959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.768616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991b384-908e-478f-8873-015ba944f5e1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.774355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.775225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.775575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.775918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.776119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b1958-27b6-4444-a8cf-8a21599f7bdf admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.776882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.781297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.781631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.782435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.782717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.783028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.784077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.785899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.810926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 939/1879] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:48.814918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.827207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b051f-5b3e-4cf6-891d-42bbbd1c02d4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.833695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:48.834436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:48.834829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:48.835209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:48.836052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:48.838025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:48.839810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 941/1880] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:48.842088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.853938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 940/1881] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:48.858367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:48.884067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecead31-74be-4053-9c8f-977f62263402 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:48.898430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:49.024244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e9e15-6b00-4799-b296-ebb83ce08a78 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:49.029779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.030467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.030750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.031071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.031752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.033998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.036244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c0c7d-c2b8-4104-8771-d430321a54e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.037282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c0c7d-c2b8-4104-8771-d430321a54e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.039882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c0c7d-c2b8-4104-8771-d430321a54e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.040391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c0c7d-c2b8-4104-8771-d430321a54e7 admin admin] PATH_INFO: `/v3/projects/22bccd71b31e47d0ab554233ed0a77f7/users/a2501bbca3c14e10a9d7a7b54d5fca1c/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.066279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 942/1882] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:48 2022] POST /identity/v3/users => generated 299 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:49.071151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:49.082561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c0c7d-c2b8-4104-8771-d430321a54e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22bccd71b31e47d0ab554233ed0a77f7, user_id=a2501bbca3c14e10a9d7a7b54d5fca1c, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.085822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c0c7d-c2b8-4104-8771-d430321a54e7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.086597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912d86-2f1d-48bf-aa47-ada2f52701c0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:49.091981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.092807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.094000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.094513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.096099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.098338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.111791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 943/1884] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:49.121203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:49.121487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 941/1884] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:48 2022] PUT /identity/v3/projects/22bccd71b31e47d0ab554233ed0a77f7/users/a2501bbca3c14e10a9d7a7b54d5fca1c/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:49.123361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434498b8-ba8c-4eb8-bd82-f9f16d4775b6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.123647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434498b8-ba8c-4eb8-bd82-f9f16d4775b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.123921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434498b8-ba8c-4eb8-bd82-f9f16d4775b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.134204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796e7c-30a9-4ab9-a155-3f0658e0235d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:49.160347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-434498b8-ba8c-4eb8-bd82-f9f16d4775b6 None None] MFA Rules not processed for user `a2501bbca3c14e10a9d7a7b54d5fca1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:49.161551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-434498b8-ba8c-4eb8-bd82-f9f16d4775b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:49.219149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.219786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.220073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.220348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] PATH_INFO: `/v3/projects/4c05dcfc10fd4c739aca8c363373a8f0/users/5b24c44769444e99af94314a08e51a57/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.239082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c05dcfc10fd4c739aca8c363373a8f0, user_id=5b24c44769444e99af94314a08e51a57, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.242899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.248956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:49.263759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 942/1885] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:49.264060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 944/1886] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:49 2022] PUT /identity/v3/projects/4c05dcfc10fd4c739aca8c363373a8f0/users/5b24c44769444e99af94314a08e51a57/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:49.267828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434498b8-ba8c-4eb8-bd82-f9f16d4775b6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:49.277218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:49.278779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:49.281173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434498b8-ba8c-4eb8-bd82-f9f16d4775b6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:49.351167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac851-532c-4bee-83e4-c853af53443d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:49.355855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.356004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.356624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.356738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.357360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.357470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.357558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.357640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.358128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.360097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.375000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 943/1887] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:49.376947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:49.387645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b72a7f-6c61-4098-ab23-e137d63dcfb6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:49.392429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e90c31-94fb-45c8-93df-a6398f080d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.393241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e90c31-94fb-45c8-93df-a6398f080d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.393530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e90c31-94fb-45c8-93df-a6398f080d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.393868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e90c31-94fb-45c8-93df-a6398f080d74 admin admin] PATH_INFO: `/v3/projects/4c05dcfc10fd4c739aca8c363373a8f0/users/5b24c44769444e99af94314a08e51a57/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.395792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e90c31-94fb-45c8-93df-a6398f080d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c05dcfc10fd4c739aca8c363373a8f0, user_id=5b24c44769444e99af94314a08e51a57, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.398751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e90c31-94fb-45c8-93df-a6398f080d74 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.413571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 944/1888] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:49 2022] PUT /identity/v3/projects/4c05dcfc10fd4c739aca8c363373a8f0/users/5b24c44769444e99af94314a08e51a57/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:49.424929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.425096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.425207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.425491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.426501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.435488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:49.436777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 945/1889] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:49.455447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] MFA Rules not processed for user `5b24c44769444e99af94314a08e51a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:49.457797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:49.525194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 945/1890] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:49.535684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:49.537060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:49.549241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a0f5f-f4fb-4f6a-8810-9d7691ad08b5 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:49.623126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca77ed-98a2-4912-8182-b71c5277062a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:49.624217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca77ed-98a2-4912-8182-b71c5277062a service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:49.624751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca77ed-98a2-4912-8182-b71c5277062a service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:49.625365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca77ed-98a2-4912-8182-b71c5277062a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:49.640289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca77ed-98a2-4912-8182-b71c5277062a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:49.642428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca77ed-98a2-4912-8182-b71c5277062a service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:49.654079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 946/1891] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:50.061386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:50.062856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:50.171107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651116b1-14ed-418b-9810-f243e72f37ce None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:50.178561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:50.179720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:50.180458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:50.180960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] PATH_INFO: `/v3/users/87733319a8404d67925cfc5c138055d9` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:50.183037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87733319a8404d67925cfc5c138055d9)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:50.189239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:50.295939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87733319a8404d67925cfc5c138055d9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.296856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87733319a8404d67925cfc5c138055d9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.299266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87733319a8404d67925cfc5c138055d9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.328033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87733319a8404d67925cfc5c138055d9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.336202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87733319a8404d67925cfc5c138055d9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.359675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 946/1892] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:50 2022] DELETE /identity/v3/users/87733319a8404d67925cfc5c138055d9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:50.364537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:50.365002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:50.365268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:50.398490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:50.399846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:50.461191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:50.480310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 947/1893] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:50.482060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:50.484757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:50.504817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067aec0-e3a3-49e6-9b28-a97d8e307c73 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:50.510756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:50.512078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:50.513519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:50.520369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:50.520687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:50.522299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] PATH_INFO: `/v3/users/3ac6ca1bb08b43228cbae335fac123a6` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:50.527979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac6ca1bb08b43228cbae335fac123a6)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:50.546669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:50.561244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:50.659840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777bac1-b7c8-4f7b-aed8-23245108f31d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:50.663136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac6ca1bb08b43228cbae335fac123a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.664422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac6ca1bb08b43228cbae335fac123a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.664620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15683fc5-cd66-41dd-930c-447eee40e9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:50.665252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15683fc5-cd66-41dd-930c-447eee40e9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:50.665533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15683fc5-cd66-41dd-930c-447eee40e9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:50.665662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac6ca1bb08b43228cbae335fac123a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.665804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15683fc5-cd66-41dd-930c-447eee40e9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:50.667639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-15683fc5-cd66-41dd-930c-447eee40e9f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:50.684722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15683fc5-cd66-41dd-930c-447eee40e9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:50.686362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15683fc5-cd66-41dd-930c-447eee40e9f4 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:50.700136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac6ca1bb08b43228cbae335fac123a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.710003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac6ca1bb08b43228cbae335fac123a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:50.716132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 948/1894] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:50.740348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 947/1895] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:50 2022] DELETE /identity/v3/users/3ac6ca1bb08b43228cbae335fac123a6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:50.742977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792401-b790-4c32-a052-9ee2599decba None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:50.744043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792401-b790-4c32-a052-9ee2599decba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:50.744423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792401-b790-4c32-a052-9ee2599decba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:50.774993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-14792401-b790-4c32-a052-9ee2599decba None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:50.776997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-14792401-b790-4c32-a052-9ee2599decba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:50.913006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 949/1896] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:50.925785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:50.927409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:51.035361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bea2e-44f5-4185-8cd9-7590f6cae3ce admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:51.041554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:51.042833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:51.043199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:51.043534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:51.053946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:51.055714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:51.067286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 948/1897] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:51.170341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14792401-b790-4c32-a052-9ee2599decba None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:51.172347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-14792401-b790-4c32-a052-9ee2599decba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:51.200343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14792401-b790-4c32-a052-9ee2599decba None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:51.209898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:51.211360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:51.211837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:51.212178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] PATH_INFO: `/v3/projects/8b26c5e0805d440a806183cffd373f28` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:51.213488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b26c5e0805d440a806183cffd373f28)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:51.215651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:51.306525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b26c5e0805d440a806183cffd373f28', 'request_id': 'req-6fbbae46-08a3-4633-834e-c4ef6b4ae709'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.315606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b26c5e0805d440a806183cffd373f28', 'request_id': 'req-6fbbae46-08a3-4633-834e-c4ef6b4ae709'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.325480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b26c5e0805d440a806183cffd373f28', 'request_id': 'req-6fbbae46-08a3-4633-834e-c4ef6b4ae709'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.338532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] The token cache is being invalidate because project 8b26c5e0805d440a806183cffd373f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:51.338825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.339875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.341321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 950/1898] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:51 2022] DELETE /identity/v3/projects/8b26c5e0805d440a806183cffd373f28 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:51.345667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:51.365052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a97d0-e251-4fee-93ad-15ae507b4e59 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:51.448403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:51.449406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:51.449873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:51.450824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] PATH_INFO: `/v3/users/7fa2fc9489504a23ab1f9f1748b18168` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:51.452347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa2fc9489504a23ab1f9f1748b18168)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:51.455147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:51.503056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:51.545392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:51.546976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:51.554217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa2fc9489504a23ab1f9f1748b18168'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.555167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa2fc9489504a23ab1f9f1748b18168'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.556194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa2fc9489504a23ab1f9f1748b18168'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.571581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa2fc9489504a23ab1f9f1748b18168'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.572548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbae46-08a3-4633-834e-c4ef6b4ae709 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:51.578909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:51.579742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:51.580139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:51.580569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] PATH_INFO: `/v3/projects/e070800e91cc46deaa9344ac32d6353a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:51.582043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e070800e91cc46deaa9344ac32d6353a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:51.584269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:51.588934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa2fc9489504a23ab1f9f1748b18168'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.608867 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 949/1899] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:51 2022] DELETE /identity/v3/users/7fa2fc9489504a23ab1f9f1748b18168 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:51.616967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:51.661572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e070800e91cc46deaa9344ac32d6353a', 'request_id': 'req-78ffcc3f-3987-453b-8c7e-01a173f008b0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.679882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e070800e91cc46deaa9344ac32d6353a', 'request_id': 'req-78ffcc3f-3987-453b-8c7e-01a173f008b0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.691641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e070800e91cc46deaa9344ac32d6353a', 'request_id': 'req-78ffcc3f-3987-453b-8c7e-01a173f008b0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.715289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] The token cache is being invalidate because project e070800e91cc46deaa9344ac32d6353a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:51.716077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.716877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.721606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 951/1900] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:51 2022] DELETE /identity/v3/projects/e070800e91cc46deaa9344ac32d6353a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:51.722940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:51.735188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:51.828486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de2b5-86a2-45ef-ade7-4d8ab6a4d67b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:51.915339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:51.916027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:51.916453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:51.916822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] PATH_INFO: `/v3/users/5b24c44769444e99af94314a08e51a57` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:51.917984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b24c44769444e99af94314a08e51a57)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:51.920322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:51.988621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b24c44769444e99af94314a08e51a57'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.989402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b24c44769444e99af94314a08e51a57'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.990111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b24c44769444e99af94314a08e51a57'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:51.997273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:52.013181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b24c44769444e99af94314a08e51a57'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.027525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d234d976-d06b-4023-9273-203dffc8f02e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b24c44769444e99af94314a08e51a57'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.047786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 950/1901] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:51 2022] DELETE /identity/v3/users/5b24c44769444e99af94314a08e51a57 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:52.048423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.050722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.067523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffcc3f-3987-453b-8c7e-01a173f008b0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.135748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.136412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.136698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.136977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] PATH_INFO: `/v3/projects/509074a028df4596b77698a521396fc0` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.138249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509074a028df4596b77698a521396fc0)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.142008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.191118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509074a028df4596b77698a521396fc0', 'request_id': 'req-0fa65927-dc10-4a18-82a6-6788d0345885'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.197830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509074a028df4596b77698a521396fc0', 'request_id': 'req-0fa65927-dc10-4a18-82a6-6788d0345885'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.205495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509074a028df4596b77698a521396fc0', 'request_id': 'req-0fa65927-dc10-4a18-82a6-6788d0345885'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.216887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] The token cache is being invalidate because project 509074a028df4596b77698a521396fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:52.217406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.218048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0fa65927-dc10-4a18-82a6-6788d0345885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.219869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 952/1902] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:52 2022] DELETE /identity/v3/projects/509074a028df4596b77698a521396fc0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:52.224822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811050-4f13-4f0d-ba1a-77ebb6a7e9f3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.225111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811050-4f13-4f0d-ba1a-77ebb6a7e9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.225423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811050-4f13-4f0d-ba1a-77ebb6a7e9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.247344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-57811050-4f13-4f0d-ba1a-77ebb6a7e9f3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:52.248473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-57811050-4f13-4f0d-ba1a-77ebb6a7e9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.309151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 951/1903] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:52.316731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398447c-815a-4162-8949-6cd11986f1fb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.316897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398447c-815a-4162-8949-6cd11986f1fb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.317101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398447c-815a-4162-8949-6cd11986f1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.340987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e398447c-815a-4162-8949-6cd11986f1fb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:52.342192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e398447c-815a-4162-8949-6cd11986f1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.350730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 953/1904] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:52.355760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57811050-4f13-4f0d-ba1a-77ebb6a7e9f3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.365569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57811050-4f13-4f0d-ba1a-77ebb6a7e9f3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.370945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.371571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.371856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.372245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.372961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:52.373274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.374880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.391034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 952/1905] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:52.392546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398447c-815a-4162-8949-6cd11986f1fb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.402631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398447c-815a-4162-8949-6cd11986f1fb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.407401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.407814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.408097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.408376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.409040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.411306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.441878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.442398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 954/1906] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:52 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:52.446576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.446708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.453223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:52.463311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd533c-2ec3-4d27-9a8d-2bd73f799d72 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.525261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.525911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.526121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.526426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] PATH_INFO: `/v3/projects/4c05dcfc10fd4c739aca8c363373a8f0` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.527696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c05dcfc10fd4c739aca8c363373a8f0)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.529618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.532517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93402-84e9-4d88-9c62-27c7644a253e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.550442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.551217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.551604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.551888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.552923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.556694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.590123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 955/1907] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:52 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:30:52.592520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.605332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c05dcfc10fd4c739aca8c363373a8f0', 'request_id': 'req-23f34b0a-c789-4f18-ad73-25b3bdcf861f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.605929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091ad91-f715-41c5-a4df-48e9a291170c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.618750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c05dcfc10fd4c739aca8c363373a8f0', 'request_id': 'req-23f34b0a-c789-4f18-ad73-25b3bdcf861f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.624568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:52.626806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c05dcfc10fd4c739aca8c363373a8f0', 'request_id': 'req-23f34b0a-c789-4f18-ad73-25b3bdcf861f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.655398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] The token cache is being invalidate because project 4c05dcfc10fd4c739aca8c363373a8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:52.655653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.657935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-23f34b0a-c789-4f18-ad73-25b3bdcf861f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.661672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 953/1908] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:52 2022] DELETE /identity/v3/projects/4c05dcfc10fd4c739aca8c363373a8f0 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:52.669745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.670827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.670827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.712004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:52.718030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.747872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.749133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.749133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.749336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.750154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.751882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.776665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 956/1909] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:52.777518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.781150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.821503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52b528-2712-4aa1-a727-4ea30bee7505 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.823502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.824396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.824926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.825419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] PATH_INFO: `/v3/users/a2501bbca3c14e10a9d7a7b54d5fca1c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.826139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 954/1910] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:52.827075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2501bbca3c14e10a9d7a7b54d5fca1c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.827441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.828904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.829664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.842577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5a3a-1386-4937-9996-9ff090e57c47 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.860968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a0e64-69ad-4f65-956d-36a5d567c8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.861723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a0e64-69ad-4f65-956d-36a5d567c8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.862078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a0e64-69ad-4f65-956d-36a5d567c8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.862420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a0e64-69ad-4f65-956d-36a5d567c8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.863512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a0e64-69ad-4f65-956d-36a5d567c8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.865768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a0e64-69ad-4f65-956d-36a5d567c8a9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.877862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 955/1911] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:52.879279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f8b2b-4392-4c1a-bbcd-49145b42be5f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.879566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f8b2b-4392-4c1a-bbcd-49145b42be5f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.879845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f8b2b-4392-4c1a-bbcd-49145b42be5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.901827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c76f8b2b-4392-4c1a-bbcd-49145b42be5f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:52.903143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c76f8b2b-4392-4c1a-bbcd-49145b42be5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.909679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2501bbca3c14e10a9d7a7b54d5fca1c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.913675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 956/1912] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:52.918268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f8b2b-4392-4c1a-bbcd-49145b42be5f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.918401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2501bbca3c14e10a9d7a7b54d5fca1c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.919113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2501bbca3c14e10a9d7a7b54d5fca1c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.920066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2501bbca3c14e10a9d7a7b54d5fca1c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.926253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f8b2b-4392-4c1a-bbcd-49145b42be5f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:52.929136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2501bbca3c14e10a9d7a7b54d5fca1c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:52.932449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52e8d2-4def-422a-a8ed-f622d7a612c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:52.933334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e8d2-4def-422a-a8ed-f622d7a612c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.933740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e8d2-4def-422a-a8ed-f622d7a612c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.934050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e8d2-4def-422a-a8ed-f622d7a612c7 admin admin] PATH_INFO: `/v3/projects/8a36f82dd64d4b399053bdf6b46868d1/users/85b2ce5f3b104c2495b9fe90f8df3f79/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:52.951543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52e8d2-4def-422a-a8ed-f622d7a612c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a36f82dd64d4b399053bdf6b46868d1, user_id=85b2ce5f3b104c2495b9fe90f8df3f79, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:52.953002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 957/1913] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:30:52 2022] DELETE /identity/v3/users/a2501bbca3c14e10a9d7a7b54d5fca1c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:52.954011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:52.954290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52e8d2-4def-422a-a8ed-f622d7a612c7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:52.955410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:52.963190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:52.972073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 957/1914] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:52 2022] PUT /identity/v3/projects/8a36f82dd64d4b399053bdf6b46868d1/users/85b2ce5f3b104c2495b9fe90f8df3f79/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:52.973478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70633778-3b61-451e-b71a-a8a20b1c088d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:52.973911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70633778-3b61-451e-b71a-a8a20b1c088d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:52.974195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70633778-3b61-451e-b71a-a8a20b1c088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.013461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-70633778-3b61-451e-b71a-a8a20b1c088d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:53.014584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-70633778-3b61-451e-b71a-a8a20b1c088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:53.065014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3916c-4a56-466b-adbf-9acde57555f0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.137022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 958/1915] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:53.138771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6851-cd3d-490c-92c1-cb5a43c79d83 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.139279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6851-cd3d-490c-92c1-cb5a43c79d83 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.139890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6851-cd3d-490c-92c1-cb5a43c79d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.150426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.151584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.152048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.152473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.153628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:53.154359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.157432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.188572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 958/1916] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:53.188725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7a6f6851-cd3d-490c-92c1-cb5a43c79d83 None None] MFA Rules not processed for user `85b2ce5f3b104c2495b9fe90f8df3f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:53.188820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.188906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7a6f6851-cd3d-490c-92c1-cb5a43c79d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:53.189557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:53.227879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:53.262103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b28df2-25e9-4b35-bf7c-db5de8d31a69 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.266857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.267698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.268383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.268757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.282165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.283166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 959/1917] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:53.284076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6851-cd3d-490c-92c1-cb5a43c79d83 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.284386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.295560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6851-cd3d-490c-92c1-cb5a43c79d83 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.297452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 959/1918] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:53.298789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.300333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:53.300773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.301399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.301785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.302067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.303125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.306406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.316171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeef2a6-cd71-4f44-b457-92fc892d4dc9 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.321751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.322484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.322951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.323298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.324032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.326020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.348601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 960/1919] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:53.353413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.364935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 960/1920] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:53.372240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.410591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:53.466950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f16ab-83da-463b-b3dc-3a3764b5b743 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.472306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683c5e-8f03-4212-a67a-0edb60f5cd45 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.477494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.478385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.478545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.478904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.479841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.482160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.506586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 961/1921] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:53.512148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.523466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd38eb-5b7a-4954-8b35-06865224f006 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.528992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.535404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.535876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.536508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.537570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.539528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.553157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.553797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.554075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.555106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.556086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.558210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.563554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 962/1922] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:53.567444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.578403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe713-86c6-4a89-9bea-f8b00dec3908 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.583646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.584363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 961/1923] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:53.584497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.584497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.584890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.585493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.587170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.588379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.598906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 963/1924] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:53.600844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cd235-6966-4108-b3ea-1cc525da9be4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.602690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.606771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.608066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.608654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.609792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.611206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.612965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4565f-3d83-4129-a9f7-4d1d03527f24 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.613730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.618662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3eaf0c-0d28-4717-95bf-4569f85f59dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.619357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3eaf0c-0d28-4717-95bf-4569f85f59dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.619632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3eaf0c-0d28-4717-95bf-4569f85f59dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.619903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3eaf0c-0d28-4717-95bf-4569f85f59dc admin admin] PATH_INFO: `/v3/projects/ed0881fedc1148fba7c2e88841ed35d2/users/411318ad6c1b4314972e14ec66da0f41/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.629032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 962/1925] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:53.633832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.635515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3eaf0c-0d28-4717-95bf-4569f85f59dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0881fedc1148fba7c2e88841ed35d2, user_id=411318ad6c1b4314972e14ec66da0f41, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.638135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3eaf0c-0d28-4717-95bf-4569f85f59dc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.644014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83cfa-80d2-417b-bbbf-d45f9926ec95 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.649247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.649859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.650131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.650403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.651247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.652817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.653640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 964/1926] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:53 2022] PUT /identity/v3/projects/ed0881fedc1148fba7c2e88841ed35d2/users/411318ad6c1b4314972e14ec66da0f41/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:30:53.665692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.666475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.666475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.675097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 963/1927] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:53.676307 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.690529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114992a-e1ef-47fe-835a-c606ba390d73 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.704591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] MFA Rules not processed for user `411318ad6c1b4314972e14ec66da0f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:53.705908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:53.728985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:53.761268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.761945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.762226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.762505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] PATH_INFO: `/v3/projects/fbd58199ac5c444795b3fed254d2efd7/users/b8a8bb4f25dd41a78d3a5a196aa1e794/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.771630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 965/1928] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:30:53.775012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.775012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:53.780284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd58199ac5c444795b3fed254d2efd7, user_id=b8a8bb4f25dd41a78d3a5a196aa1e794, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.783087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.787997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47375d-aaca-4656-ab98-328507711d18 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.794027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.794757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.795075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.795350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] PATH_INFO: `/v3/projects/22bccd71b31e47d0ab554233ed0a77f7` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.796513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22bccd71b31e47d0ab554233ed0a77f7)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.797633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 964/1929] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:53 2022] PUT /identity/v3/projects/fbd58199ac5c444795b3fed254d2efd7/users/b8a8bb4f25dd41a78d3a5a196aa1e794/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:53.798416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:53.799734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.864541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22bccd71b31e47d0ab554233ed0a77f7', 'request_id': 'req-549f1ba0-f759-4b77-8772-2e04524ad2c6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:53.868022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:53.873878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22bccd71b31e47d0ab554233ed0a77f7', 'request_id': 'req-549f1ba0-f759-4b77-8772-2e04524ad2c6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:53.881508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344471c-bba8-41ab-baea-9e9069ff3a21 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:53.887185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22bccd71b31e47d0ab554233ed0a77f7', 'request_id': 'req-549f1ba0-f759-4b77-8772-2e04524ad2c6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:53.899839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] The token cache is being invalidate because project 22bccd71b31e47d0ab554233ed0a77f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:30:53.900130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:53.900880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-549f1ba0-f759-4b77-8772-2e04524ad2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:30:53.902468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 966/1930] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:30:53 2022] DELETE /identity/v3/projects/22bccd71b31e47d0ab554233ed0a77f7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:53.904147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.904437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.904716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.936205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] MFA Rules not processed for user `b8a8bb4f25dd41a78d3a5a196aa1e794`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:53.937429 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:53.958741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07d26c-631e-43bf-9bf7-3ca36c17f23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:53.960032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07d26c-631e-43bf-9bf7-3ca36c17f23c admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.960340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07d26c-631e-43bf-9bf7-3ca36c17f23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.960650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07d26c-631e-43bf-9bf7-3ca36c17f23c admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:53.961316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07d26c-631e-43bf-9bf7-3ca36c17f23c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:53.963405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07d26c-631e-43bf-9bf7-3ca36c17f23c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:53.997610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 965/1931] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:53.999042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:53.999336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:53.999618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.025836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:54.026900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 967/1932] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:54.027129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:54.030987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.032352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:54.046863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d52c-34b0-404d-b664-1bb075e249b2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.068952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:54.094092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 966/1933] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:54.094092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.094092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:54.109638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f6bd-52b1-493c-b72c-225c36b5126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.110266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f6bd-52b1-493c-b72c-225c36b5126b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.110547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f6bd-52b1-493c-b72c-225c36b5126b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.110823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f6bd-52b1-493c-b72c-225c36b5126b admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.111517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f6bd-52b1-493c-b72c-225c36b5126b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.113443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f6bd-52b1-493c-b72c-225c36b5126b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.135699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 968/1934] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:54 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:54.136977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775bfa-1a12-4112-9b77-64d2b4c34985 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.137293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775bfa-1a12-4112-9b77-64d2b4c34985 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.137572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775bfa-1a12-4112-9b77-64d2b4c34985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.156953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-16775bfa-1a12-4112-9b77-64d2b4c34985 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:54.158157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-16775bfa-1a12-4112-9b77-64d2b4c34985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:54.167508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 969/1935] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:54.167859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775bfa-1a12-4112-9b77-64d2b4c34985 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.175989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcedd8f-d6f2-4498-bd73-9bdc50c8f149 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.177978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775bfa-1a12-4112-9b77-64d2b4c34985 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.179908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.180785 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.181066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.181399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.182903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.183514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.183941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.184127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.184802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.186484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.202163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 970/1936] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:54.202163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.211717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3be13-e88b-47e9-a28c-f401638fda25 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.216334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.216942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.217219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.217963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.218173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:30:54.218479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.220099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.233794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 971/1937] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:30:54.234931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.246325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4b87d-3ba1-4069-a101-6d1bb93b016a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.251255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.251931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.252211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.252486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] PATH_INFO: `/v3/projects/0c68a46c8abc47b38def2972017e5c8d/users/2b471b09195d4371bb064bc26a586c4c/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.256410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.258367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.269639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c68a46c8abc47b38def2972017e5c8d, user_id=2b471b09195d4371bb064bc26a586c4c, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.272333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.272841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 967/1938] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:54.274923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.294507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88fb5f-aa97-436f-95a4-23ed697a4d9a service glance] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.299015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 972/1939] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:54 2022] PUT /identity/v3/projects/0c68a46c8abc47b38def2972017e5c8d/users/2b471b09195d4371bb064bc26a586c4c/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:54.302619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.311832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:54.381413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266aa1f6-3f72-41c1-b174-d6c37f8a73b8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.396459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.397099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.397397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.397654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.399087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.400786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.415623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 973/1940] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:54.416716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.429214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1d14b-30a1-4a93-8977-6a6e9a3bef57 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.441772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.442944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.443997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.444382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.446044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.448848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de37402-876e-486b-b4a4-89226e684ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.448986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.451503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de37402-876e-486b-b4a4-89226e684ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.451801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de37402-876e-486b-b4a4-89226e684ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.452078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de37402-876e-486b-b4a4-89226e684ee0 admin admin] PATH_INFO: `/v3/projects/0c68a46c8abc47b38def2972017e5c8d/users/2b471b09195d4371bb064bc26a586c4c/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.457888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de37402-876e-486b-b4a4-89226e684ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c68a46c8abc47b38def2972017e5c8d, user_id=2b471b09195d4371bb064bc26a586c4c, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.461131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de37402-876e-486b-b4a4-89226e684ee0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.498616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 968/1941] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:30:54 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:54.500635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.518763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 974/1942] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:54 2022] PUT /identity/v3/projects/0c68a46c8abc47b38def2972017e5c8d/users/2b471b09195d4371bb064bc26a586c4c/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:54.519236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:54.525627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.525920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.526206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.589731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] MFA Rules not processed for user `2b471b09195d4371bb064bc26a586c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:54.591309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:54.601550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4748-a4d1-438b-9e46-f20eacc99689 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.608017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.608738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.609088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.610166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.610967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.613384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.644026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 969/1943] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:30:54 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:30:54.649214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.667165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e51279-5995-4a20-a11c-97e33afc641e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.677604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.678349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.678841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.679413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.681736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.685358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.703367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 970/1944] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:30:54.708591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.733189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca6e5-64af-44b2-bcef-41f4bff64700 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.747507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.749168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.749519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.749933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.751937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.755138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.762793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 975/1945] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:54.769343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 971/1946] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:30:54.774169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.774615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:54.775926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:54.803073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570826d8-3b26-4ecf-a1be-cfb51eebc9a4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.814300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b8993-1832-400a-b01f-e12da6492ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.815031 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b8993-1832-400a-b01f-e12da6492ded admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.815316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b8993-1832-400a-b01f-e12da6492ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.815594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b8993-1832-400a-b01f-e12da6492ded admin admin] PATH_INFO: `/v3/projects/1d02ee1ad57f4d38acf0bc95ab43f614/users/de557945688e4d9f944310015bb42163/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.856320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b8993-1832-400a-b01f-e12da6492ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d02ee1ad57f4d38acf0bc95ab43f614, user_id=de557945688e4d9f944310015bb42163, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:54.860886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b8993-1832-400a-b01f-e12da6492ded admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:54.879404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 972/1947] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:30:54 2022] PUT /identity/v3/projects/1d02ee1ad57f4d38acf0bc95ab43f614/users/de557945688e4d9f944310015bb42163/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:30:54.885206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d6bb-3ed4-4ffe-8e7a-6561eece9e08 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:54.892710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.893102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.893973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:54.940506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] MFA Rules not processed for user `de557945688e4d9f944310015bb42163`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:54.941987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:54.942496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:30:54.969513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:54.970239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:54.970525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:54.970801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:55.018972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 973/1948] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:30:55.030552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:55.031930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:55.036398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:55.037976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:55.045413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b25ad3-d9bb-460d-9f1f-bf20338f9e02 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:55.048027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 976/1949] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:55.049439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414e3ec-fa3a-4509-b97b-d4f21dcb8518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:55.050185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e3ec-fa3a-4509-b97b-d4f21dcb8518 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:55.050506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e3ec-fa3a-4509-b97b-d4f21dcb8518 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:55.051280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e3ec-fa3a-4509-b97b-d4f21dcb8518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:55.061761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414e3ec-fa3a-4509-b97b-d4f21dcb8518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:55.063301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414e3ec-fa3a-4509-b97b-d4f21dcb8518 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:55.073784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 974/1950] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:30:55.775082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:30:55.786904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a01363-aacf-46d7-9821-8f376614acfe service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:30:55.788339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:30:55.789512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:55.789792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:55.790061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:55.791854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:55.876571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:30:55.878172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:30:55.889046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 977/1951] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:30:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:30:56.589335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772d2d5-bdcf-4b23-b9e4-0ffbc1bad9d7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:30:56.589898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772d2d5-bdcf-4b23-b9e4-0ffbc1bad9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:30:56.589898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772d2d5-bdcf-4b23-b9e4-0ffbc1bad9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:30:56.612885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1772d2d5-bdcf-4b23-b9e4-0ffbc1bad9d7 None None] MFA Rules not processed for user `de557945688e4d9f944310015bb42163`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:30:56.614129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1772d2d5-bdcf-4b23-b9e4-0ffbc1bad9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:30:56.625494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 975/1952] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:30:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:00.218344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:00.228388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7a767-468e-4dda-81a0-86ea0d60f858 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:00.232042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07d983-ac11-4251-bc5b-d4b8dd90ed6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:00.232658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07d983-ac11-4251-bc5b-d4b8dd90ed6d service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:00.232958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07d983-ac11-4251-bc5b-d4b8dd90ed6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:00.233228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07d983-ac11-4251-bc5b-d4b8dd90ed6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:00.235008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9b07d983-ac11-4251-bc5b-d4b8dd90ed6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:00.292672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07d983-ac11-4251-bc5b-d4b8dd90ed6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:00.294121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07d983-ac11-4251-bc5b-d4b8dd90ed6d service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:00.303278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 978/1953] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:02.579730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772d2d5-bdcf-4b23-b9e4-0ffbc1bad9d7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:02.613961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:02.675393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772d2d5-bdcf-4b23-b9e4-0ffbc1bad9d7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:02.681417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:02.682158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:02.682548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:02.682963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] PATH_INFO: `/v3/users/85b2ce5f3b104c2495b9fe90f8df3f79` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:02.684792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b2ce5f3b104c2495b9fe90f8df3f79)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:02.687171 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:02.748107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b2ce5f3b104c2495b9fe90f8df3f79'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:02.749049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b2ce5f3b104c2495b9fe90f8df3f79'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:02.749733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b2ce5f3b104c2495b9fe90f8df3f79'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:02.773635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b2ce5f3b104c2495b9fe90f8df3f79'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:02.782046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b2ce5f3b104c2495b9fe90f8df3f79'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:02.798929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 976/1954] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:02 2022] DELETE /identity/v3/users/85b2ce5f3b104c2495b9fe90f8df3f79 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:02.803370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:02.804824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:02.805105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:02.828742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:02.830260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:02.862284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:02.932280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 979/1955] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:02.943701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:02.945269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:03.050294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9af0-c8e9-4524-99ba-1d79f575c76a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:03.063964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:03.064626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:03.064913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:03.065235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:03.079770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:03.081533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:03.096693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 977/1956] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:03.518418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:03.520244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:03.534002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb8e90-d5d6-45dc-af13-73d2dcea4bd3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:03.539115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:03.539770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:03.540052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:03.540327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] PATH_INFO: `/v3/projects/8a36f82dd64d4b399053bdf6b46868d1` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:03.541852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a36f82dd64d4b399053bdf6b46868d1)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:03.543848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:03.599248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a36f82dd64d4b399053bdf6b46868d1', 'request_id': 'req-ec6f1438-ebe8-4074-ac27-82834f7d61c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.605925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a36f82dd64d4b399053bdf6b46868d1', 'request_id': 'req-ec6f1438-ebe8-4074-ac27-82834f7d61c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.613827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a36f82dd64d4b399053bdf6b46868d1', 'request_id': 'req-ec6f1438-ebe8-4074-ac27-82834f7d61c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.626558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] The token cache is being invalidate because project 8a36f82dd64d4b399053bdf6b46868d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:03.626849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.627536 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.629077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 980/1957] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:03 2022] DELETE /identity/v3/projects/8a36f82dd64d4b399053bdf6b46868d1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:03.633242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:03.642995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312359a-0c68-45ab-a567-6efd6632bea9 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:03.684732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:03.705976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:03.706743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:03.707058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:03.707353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] PATH_INFO: `/v3/users/411318ad6c1b4314972e14ec66da0f41` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:03.708504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411318ad6c1b4314972e14ec66da0f41)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:03.710434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:03.761662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411318ad6c1b4314972e14ec66da0f41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.762340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411318ad6c1b4314972e14ec66da0f41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.763011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411318ad6c1b4314972e14ec66da0f41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.776683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411318ad6c1b4314972e14ec66da0f41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.783865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411318ad6c1b4314972e14ec66da0f41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:03.798077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 978/1958] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:03 2022] DELETE /identity/v3/users/411318ad6c1b4314972e14ec66da0f41 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:04.168410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:04.170199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:04.211085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f1438-ebe8-4074-ac27-82834f7d61c0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:04.322444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:04.322444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:04.322444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:04.322444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] PATH_INFO: `/v3/projects/ed0881fedc1148fba7c2e88841ed35d2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:04.324112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0881fedc1148fba7c2e88841ed35d2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:04.326152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:04.374039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:04.383559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0881fedc1148fba7c2e88841ed35d2', 'request_id': 'req-8a45e112-8ed5-49f3-abe8-f379d8a2fead'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.390741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0881fedc1148fba7c2e88841ed35d2', 'request_id': 'req-8a45e112-8ed5-49f3-abe8-f379d8a2fead'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.401425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0881fedc1148fba7c2e88841ed35d2', 'request_id': 'req-8a45e112-8ed5-49f3-abe8-f379d8a2fead'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.413950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] The token cache is being invalidate because project ed0881fedc1148fba7c2e88841ed35d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:04.414244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.415111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.416955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 981/1959] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:04 2022] DELETE /identity/v3/projects/ed0881fedc1148fba7c2e88841ed35d2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:04.420819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:04.422246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:04.437149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01ef1a-3457-479b-80fa-3ae646e0f8f9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:04.503173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:04.503893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:04.504181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:04.504461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] PATH_INFO: `/v3/users/2b471b09195d4371bb064bc26a586c4c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:04.507562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b471b09195d4371bb064bc26a586c4c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:04.509903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:04.595211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b471b09195d4371bb064bc26a586c4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.597010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b471b09195d4371bb064bc26a586c4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.597655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b471b09195d4371bb064bc26a586c4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.609839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:04.612126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b471b09195d4371bb064bc26a586c4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.622774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2819e50-5deb-4434-91a1-965fa62c2b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b471b09195d4371bb064bc26a586c4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:04.644032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 979/1960] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:04 2022] DELETE /identity/v3/users/2b471b09195d4371bb064bc26a586c4c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:04.954699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:04.956238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:04.978029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45e112-8ed5-49f3-abe8-f379d8a2fead admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:05.056247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:05.056973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.057271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.057586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] PATH_INFO: `/v3/projects/0c68a46c8abc47b38def2972017e5c8d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.058834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c68a46c8abc47b38def2972017e5c8d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:05.064135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:05.114099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c68a46c8abc47b38def2972017e5c8d', 'request_id': 'req-8c7f5407-82d6-4a70-8596-26d01dec6189'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:05.121318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c68a46c8abc47b38def2972017e5c8d', 'request_id': 'req-8c7f5407-82d6-4a70-8596-26d01dec6189'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:05.137031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c68a46c8abc47b38def2972017e5c8d', 'request_id': 'req-8c7f5407-82d6-4a70-8596-26d01dec6189'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:05.151089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] The token cache is being invalidate because project 0c68a46c8abc47b38def2972017e5c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:05.151387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:05.152119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c7f5407-82d6-4a70-8596-26d01dec6189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:05.157932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 982/1961] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:04 2022] DELETE /identity/v3/projects/0c68a46c8abc47b38def2972017e5c8d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:05.159921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7325a-c065-412d-b8c0-003644719f38 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.160215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7325a-c065-412d-b8c0-003644719f38 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.160497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7325a-c065-412d-b8c0-003644719f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.181419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cda7325a-c065-412d-b8c0-003644719f38 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:05.182626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cda7325a-c065-412d-b8c0-003644719f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:05.261559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 980/1962] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:05.262715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfa2b2-ea9f-417b-80d6-4b4d08e22fd0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.263201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfa2b2-ea9f-417b-80d6-4b4d08e22fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.263303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfa2b2-ea9f-417b-80d6-4b4d08e22fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.296111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a9bfa2b2-ea9f-417b-80d6-4b4d08e22fd0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:05.297465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a9bfa2b2-ea9f-417b-80d6-4b4d08e22fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:05.310202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 983/1963] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:05.314094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7325a-c065-412d-b8c0-003644719f38 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:05.325353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7325a-c065-412d-b8c0-003644719f38 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:05.330467 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:05.337099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.338305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.338599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.339421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:05.340186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:05.341379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:05.367986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 981/1964] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:31:05.373724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfa2b2-ea9f-417b-80d6-4b4d08e22fd0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:05.384040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfa2b2-ea9f-417b-80d6-4b4d08e22fd0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:05.389584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:05.390236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.390521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.390803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.391545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:05.393490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:05.437469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 984/1965] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:05 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:05.441643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:05.465517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb07d8-696d-4042-ba3c-f1f4590fc378 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:05.564652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:05.565581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.565941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.566296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.567272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:05.569760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:05.609692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 982/1966] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:05 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:05.613927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:05.626149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd5b9e-a184-4dc8-a920-dc3fc40399b7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:05.631400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:05.632698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.632869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.633109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.633788 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:05.635534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:05.649036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 985/1967] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:05.653048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:05.663826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e736649-2bb6-4a3a-b962-67e994e188b4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:05.670247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69f9d2-d0b1-4015-b126-ca75179a97b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:05.671041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69f9d2-d0b1-4015-b126-ca75179a97b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.671425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69f9d2-d0b1-4015-b126-ca75179a97b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.671854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69f9d2-d0b1-4015-b126-ca75179a97b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.672611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69f9d2-d0b1-4015-b126-ca75179a97b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:05.674308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69f9d2-d0b1-4015-b126-ca75179a97b4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:05.690406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 983/1968] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:05.691664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:05.702624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b349d7-b34f-43e4-8792-220905efedf4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:05.707554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:05.708404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.708748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.709095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] PATH_INFO: `/v3/projects/2bcb093c643e4b72aa6af6141b09f6cb/users/05ec72465df84095a284ba419ae802ee/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.729515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bcb093c643e4b72aa6af6141b09f6cb, user_id=05ec72465df84095a284ba419ae802ee, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:05.732527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:05.736505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:05.759795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 986/1969] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:05 2022] PUT /identity/v3/projects/2bcb093c643e4b72aa6af6141b09f6cb/users/05ec72465df84095a284ba419ae802ee/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:05.773587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d0fec-6bee-49b4-8bb1-67052ccd8e32 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:05.773898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d0fec-6bee-49b4-8bb1-67052ccd8e32 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:05.774137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d0fec-6bee-49b4-8bb1-67052ccd8e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:05.819259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-697d0fec-6bee-49b4-8bb1-67052ccd8e32 None None] MFA Rules not processed for user `05ec72465df84095a284ba419ae802ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:05.821210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-697d0fec-6bee-49b4-8bb1-67052ccd8e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:05.872345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:05.953565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 984/1970] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:05.959032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:06.119436 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379bbb-2e2b-4b11-bd3f-ba5e3c290bd7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:06.130707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:06.133148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.133433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.135340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.136121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:06.138653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:06.196279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 987/1971] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:05 2022] POST /identity/v3/projects => generated 399 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:06.200568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d0fec-6bee-49b4-8bb1-67052ccd8e32 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:06.212281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d0fec-6bee-49b4-8bb1-67052ccd8e32 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:06.275278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:06.275925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.276214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.276495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.277194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:06.279167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:06.311412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 985/1972] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:06 2022] POST /identity/v3/users => generated 306 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:06.315151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:06.327274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bf5c0-947b-47ae-8cc6-6a3bed212c2f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:06.332587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:06.333231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.333514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.333788 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.334488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:06.336252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:06.348966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 988/1973] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:31:06.353837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:06.365977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd2d8d-30cc-45f5-864c-fd7279c5ee2f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:06.371306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:06.372021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.372301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.372577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] PATH_INFO: `/v3/projects/98d5facc35a1442d8acb4ef072b5760a/users/87aada89fda240faa9fdba7175472d01/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.390584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d5facc35a1442d8acb4ef072b5760a, user_id=87aada89fda240faa9fdba7175472d01, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:06.393381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:06.410742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 986/1974] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:06 2022] PUT /identity/v3/projects/98d5facc35a1442d8acb4ef072b5760a/users/87aada89fda240faa9fdba7175472d01/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:06.415333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:06.428421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e78a5b-54dd-4a32-a578-20c247a8f4b7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:06.500330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089a390-d32f-42a5-bcf2-34b26bf306ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:06.500851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089a390-d32f-42a5-bcf2-34b26bf306ec admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.501133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089a390-d32f-42a5-bcf2-34b26bf306ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.501778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089a390-d32f-42a5-bcf2-34b26bf306ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.502111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089a390-d32f-42a5-bcf2-34b26bf306ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:06.503771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089a390-d32f-42a5-bcf2-34b26bf306ec admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:06.519896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 989/1975] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:06.521116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:06.539999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b51242-e318-44b7-adfb-4a7b00e4130c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:06.545839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:06.546614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.546978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.547873 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] PATH_INFO: `/v3/projects/98d5facc35a1442d8acb4ef072b5760a/users/87aada89fda240faa9fdba7175472d01/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.550573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d5facc35a1442d8acb4ef072b5760a, user_id=87aada89fda240faa9fdba7175472d01, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:06.555990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:06.570106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 987/1976] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:06 2022] PUT /identity/v3/projects/98d5facc35a1442d8acb4ef072b5760a/users/87aada89fda240faa9fdba7175472d01/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:06.585790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7f103-5c2e-4d3f-9358-c9840f6c5b26 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.586191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7f103-5c2e-4d3f-9358-c9840f6c5b26 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.586573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7f103-5c2e-4d3f-9358-c9840f6c5b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.635751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f3a7f103-5c2e-4d3f-9358-c9840f6c5b26 None None] MFA Rules not processed for user `87aada89fda240faa9fdba7175472d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:06.636984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f3a7f103-5c2e-4d3f-9358-c9840f6c5b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:06.720206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:06.746906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 990/1977] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:06.759409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:06.762021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:06.864790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f597a-8d8b-4ab9-bf0c-3ca9eda5d8c5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:06.868902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:06.869542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:06.869828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:06.870103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:06.880397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:06.881828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:06.895720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 988/1978] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:07.036291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7f103-5c2e-4d3f-9358-c9840f6c5b26 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:07.047454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7f103-5c2e-4d3f-9358-c9840f6c5b26 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:07.051663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5fd5b-84d4-4db7-bb4b-4cf049906d67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:07.052393 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5fd5b-84d4-4db7-bb4b-4cf049906d67 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:07.052765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5fd5b-84d4-4db7-bb4b-4cf049906d67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:07.053140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5fd5b-84d4-4db7-bb4b-4cf049906d67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:07.117956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5fd5b-84d4-4db7-bb4b-4cf049906d67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:07.119272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5fd5b-84d4-4db7-bb4b-4cf049906d67 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:07.135048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 991/1979] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:12.499946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:12.530331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:12.566129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47821-86ba-4e35-ad31-3d6b18161645 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:12.571034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:12.571679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:12.571961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:12.572282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] PATH_INFO: `/v3/users/05ec72465df84095a284ba419ae802ee` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:12.573409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ec72465df84095a284ba419ae802ee)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:12.575542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:12.634450 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ec72465df84095a284ba419ae802ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:12.635333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ec72465df84095a284ba419ae802ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:12.635938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ec72465df84095a284ba419ae802ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:12.648349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ec72465df84095a284ba419ae802ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:12.655686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ec72465df84095a284ba419ae802ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:12.669114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 989/1980] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:12 2022] DELETE /identity/v3/users/05ec72465df84095a284ba419ae802ee => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:12.673569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:12.674048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:12.674185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:12.693928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:12.695214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:12.760473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 992/1981] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:12.767640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:12.769980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:12.843362 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eee64-f32a-4ee4-b286-97bb614bcfbe admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:12.846740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:12.847466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:12.847748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:12.848020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:12.861918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:12.863595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:12.874668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 990/1982] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:13.187825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:13.197137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:13.214631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843e3de-93c4-4d0d-a8db-94dfd9e9c789 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:13.227792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:13.228456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:13.228735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:13.229008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] PATH_INFO: `/v3/projects/2bcb093c643e4b72aa6af6141b09f6cb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:13.230453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bcb093c643e4b72aa6af6141b09f6cb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:13.232577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:13.292542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bcb093c643e4b72aa6af6141b09f6cb', 'request_id': 'req-82693569-b36f-44e5-a205-3da02aeb79bd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.298995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bcb093c643e4b72aa6af6141b09f6cb', 'request_id': 'req-82693569-b36f-44e5-a205-3da02aeb79bd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.311693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bcb093c643e4b72aa6af6141b09f6cb', 'request_id': 'req-82693569-b36f-44e5-a205-3da02aeb79bd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.329809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] The token cache is being invalidate because project 2bcb093c643e4b72aa6af6141b09f6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:13.330149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.330837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.333490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 993/1983] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:13 2022] DELETE /identity/v3/projects/2bcb093c643e4b72aa6af6141b09f6cb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:13.338613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:13.349896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fe96a-b736-4267-ab3b-37976b581a76 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:13.411787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:13.423815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:13.424724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:13.425025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:13.426228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] PATH_INFO: `/v3/users/87aada89fda240faa9fdba7175472d01` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:13.427444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] RBAC: Authorizing `identity:delete_user(user_id=87aada89fda240faa9fdba7175472d01)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:13.429839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:13.485382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87aada89fda240faa9fdba7175472d01'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.486047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87aada89fda240faa9fdba7175472d01'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.486652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87aada89fda240faa9fdba7175472d01'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.499336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87aada89fda240faa9fdba7175472d01'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.508494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a166cfb-831c-4805-a7ee-f32ce377965b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87aada89fda240faa9fdba7175472d01'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:13.523223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 991/1984] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:13 2022] DELETE /identity/v3/users/87aada89fda240faa9fdba7175472d01 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:13.860031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:13.861595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:13.876555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82693569-b36f-44e5-a205-3da02aeb79bd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:13.890220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:13.953711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:13.954640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:13.955133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:13.955631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] PATH_INFO: `/v3/projects/98d5facc35a1442d8acb4ef072b5760a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:13.957059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d5facc35a1442d8acb4ef072b5760a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:13.960645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.019654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d5facc35a1442d8acb4ef072b5760a', 'request_id': 'req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:14.027450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d5facc35a1442d8acb4ef072b5760a', 'request_id': 'req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:14.035236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d5facc35a1442d8acb4ef072b5760a', 'request_id': 'req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:14.047508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] The token cache is being invalidate because project 98d5facc35a1442d8acb4ef072b5760a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:14.047871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:14.048571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1d9ba2d5-28e9-450a-93de-9ac5b39bb869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:14.050315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 994/1985] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:13 2022] DELETE /identity/v3/projects/98d5facc35a1442d8acb4ef072b5760a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:14.055982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f44e2-305b-4b32-a1cc-041fbcbee705 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.056271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f44e2-305b-4b32-a1cc-041fbcbee705 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.056549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f44e2-305b-4b32-a1cc-041fbcbee705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.075704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f15f44e2-305b-4b32-a1cc-041fbcbee705 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:14.076847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f15f44e2-305b-4b32-a1cc-041fbcbee705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:14.139376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 992/1986] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:14.147846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adc52f-0a5a-4316-940a-4895671e89e6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.148137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adc52f-0a5a-4316-940a-4895671e89e6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.148415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adc52f-0a5a-4316-940a-4895671e89e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.168198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b6adc52f-0a5a-4316-940a-4895671e89e6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:14.169405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b6adc52f-0a5a-4316-940a-4895671e89e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:14.177291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 995/1987] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:14.182408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f44e2-305b-4b32-a1cc-041fbcbee705 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.192304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f44e2-305b-4b32-a1cc-041fbcbee705 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.197174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.197868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.198240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.198553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.199360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:14.199803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.201618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.218606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 993/1988] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:31:14.219894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adc52f-0a5a-4316-940a-4895671e89e6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.230774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adc52f-0a5a-4316-940a-4895671e89e6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.235932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.236633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.237109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.237447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.238180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.240460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.276833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 996/1989] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:14 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:14.278006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.288631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e9d41-54dc-439a-9a78-0bb2a0701bf1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.343111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:14.353484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.354209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.354499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.354784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.355600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.357909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.382915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 994/1990] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:14 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:14.385921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.397110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3263708-ecc7-4cac-9ea1-1db3bf996727 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.403188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.403929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.404288 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.404637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.405589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.407140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.423598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 997/1991] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:14.427763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.440846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edf3f6-7afb-4266-82c9-309e907c6d31 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.447405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993cd3-d14a-43a5-b289-a28318412d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.448152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993cd3-d14a-43a5-b289-a28318412d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.448492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993cd3-d14a-43a5-b289-a28318412d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.448826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993cd3-d14a-43a5-b289-a28318412d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.449718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de993cd3-d14a-43a5-b289-a28318412d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.451594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de993cd3-d14a-43a5-b289-a28318412d58 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.463864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 995/1992] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:31:14.467967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.478715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec815635-fa97-401d-a7a7-846ad53315d2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.484819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.485657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.485945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.486220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] PATH_INFO: `/v3/projects/76e75b8ff2774a8785df62db09a15c01/users/38bf0ab171ee4af5b4d2b0f1302d2383/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.501853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e75b8ff2774a8785df62db09a15c01, user_id=38bf0ab171ee4af5b4d2b0f1302d2383, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.505833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.522911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 998/1993] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:14 2022] PUT /identity/v3/projects/76e75b8ff2774a8785df62db09a15c01/users/38bf0ab171ee4af5b4d2b0f1302d2383/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:14.531431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8039186-e84f-40a5-bfcf-5fe9f9788fae None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.531971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8039186-e84f-40a5-bfcf-5fe9f9788fae None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.532179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8039186-e84f-40a5-bfcf-5fe9f9788fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.565029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a8039186-e84f-40a5-bfcf-5fe9f9788fae None None] MFA Rules not processed for user `38bf0ab171ee4af5b4d2b0f1302d2383`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:14.566413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a8039186-e84f-40a5-bfcf-5fe9f9788fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:14.642192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 996/1994] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:14.646559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.694965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:14.716710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f896a9-1e8b-4d7c-b9fb-780e278a8c44 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.721404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.722023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.722304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.722580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.723273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.725206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.751540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 999/1995] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:14 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:14.755631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8039186-e84f-40a5-bfcf-5fe9f9788fae None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.766743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8039186-e84f-40a5-bfcf-5fe9f9788fae None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.822596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.823246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.823529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.823806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.824466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.826388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.850184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 997/1996] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:14 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:14.854217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.864771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60675c9-fe05-4704-830a-2a9100916c00 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.869455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.870084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.870361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.870632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.871343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.872935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.883783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1000/1997] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:14.887768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.898200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbdb35-1d2d-4e00-9afd-ddedf2587e8a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:14.902762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:14.903442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:14.903719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:14.903995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] PATH_INFO: `/v3/projects/843afdacd2f1482f9a0f283a84720f45/users/3a78167420fa4daf8856ebfc091ebf70/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:14.918545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=843afdacd2f1482f9a0f283a84720f45, user_id=3a78167420fa4daf8856ebfc091ebf70, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:14.921273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:14.934968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 998/1998] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:14 2022] PUT /identity/v3/projects/843afdacd2f1482f9a0f283a84720f45/users/3a78167420fa4daf8856ebfc091ebf70/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:14.939046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:14.949794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f688d9f-edc1-4afc-8899-82add10f757e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:15.004605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc36d9-13b7-4354-9f37-45fb9b1a14d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:15.005582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc36d9-13b7-4354-9f37-45fb9b1a14d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:15.005886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc36d9-13b7-4354-9f37-45fb9b1a14d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:15.006154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc36d9-13b7-4354-9f37-45fb9b1a14d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:15.007361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc36d9-13b7-4354-9f37-45fb9b1a14d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:15.009054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc36d9-13b7-4354-9f37-45fb9b1a14d0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:15.019059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1001/1999] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:15.023946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:15.034051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b478889-403c-4dab-aaa0-fa52edea6b5f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:15.038680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:15.039383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:15.039664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:15.039938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] PATH_INFO: `/v3/projects/843afdacd2f1482f9a0f283a84720f45/users/3a78167420fa4daf8856ebfc091ebf70/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:15.041898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843afdacd2f1482f9a0f283a84720f45, user_id=3a78167420fa4daf8856ebfc091ebf70, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:15.044835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:15.055765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 999/2000] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:15 2022] PUT /identity/v3/projects/843afdacd2f1482f9a0f283a84720f45/users/3a78167420fa4daf8856ebfc091ebf70/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:15.066811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:15.067229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:15.067513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:15.096636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] MFA Rules not processed for user `3a78167420fa4daf8856ebfc091ebf70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:15.097830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:15.162951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1002/2001] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:15.174608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:15.176487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:15.199476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:15.236723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740baf74-229f-4244-bf74-c7d9bdc2fed0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:15.240357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:15.240973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:15.241755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:15.242109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:15.251226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:15.252641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:15.262607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1000/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:15.363638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:15.365126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:15.377696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcfd10-535a-4626-a0e1-a23f9be24db6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:15.381480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5ba7f-7963-4557-913b-f8addd90e13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:15.382763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ba7f-7963-4557-913b-f8addd90e13a service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:15.383074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ba7f-7963-4557-913b-f8addd90e13a service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:15.383354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ba7f-7963-4557-913b-f8addd90e13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:15.416089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:15.442505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5ba7f-7963-4557-913b-f8addd90e13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:15.444424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5ba7f-7963-4557-913b-f8addd90e13a service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:15.454275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1003/2003] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:19.136883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:19.198620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f0e9c-8c3e-420f-87f3-f2e3c079840a service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:19.203131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:19.203761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:19.204037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:19.204309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] PATH_INFO: `/v3/users/38bf0ab171ee4af5b4d2b0f1302d2383` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:19.205510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38bf0ab171ee4af5b4d2b0f1302d2383)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:19.207446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:19.252712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38bf0ab171ee4af5b4d2b0f1302d2383'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.253515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38bf0ab171ee4af5b4d2b0f1302d2383'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.254203 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38bf0ab171ee4af5b4d2b0f1302d2383'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.268538 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38bf0ab171ee4af5b4d2b0f1302d2383'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.276558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38bf0ab171ee4af5b4d2b0f1302d2383'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.290653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1001/2004] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:19 2022] DELETE /identity/v3/users/38bf0ab171ee4af5b4d2b0f1302d2383 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:19.295150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:19.295680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:19.295857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:19.316489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:19.317777 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:19.383127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1004/2005] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:19.392895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:19.394361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:19.438942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:19.455111 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e986-c500-44f3-800b-9dca9fc896c2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:19.458693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:19.459345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:19.459648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:19.460555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:19.469285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:19.470691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:19.479990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1002/2006] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:19.782230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:19.783761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:19.796418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfd0f7-e663-41c4-b0c3-79d775f2f9a9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:19.801119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:19.801767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:19.802050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:19.802324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] PATH_INFO: `/v3/projects/76e75b8ff2774a8785df62db09a15c01` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:19.804050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e75b8ff2774a8785df62db09a15c01)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:19.806221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:19.853872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e75b8ff2774a8785df62db09a15c01', 'request_id': 'req-aeaae14e-1d85-4ab8-9184-441a387f6ce9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.860111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e75b8ff2774a8785df62db09a15c01', 'request_id': 'req-aeaae14e-1d85-4ab8-9184-441a387f6ce9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.867353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e75b8ff2774a8785df62db09a15c01', 'request_id': 'req-aeaae14e-1d85-4ab8-9184-441a387f6ce9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.877746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] The token cache is being invalidate because project 76e75b8ff2774a8785df62db09a15c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:19.878037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.878681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.880246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1005/2007] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:19 2022] DELETE /identity/v3/projects/76e75b8ff2774a8785df62db09a15c01 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:19.884829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:19.894457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dab27-c20f-4c66-831f-73d4795a4e4b service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:19.950357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:19.951025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:19.951306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:19.951578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] PATH_INFO: `/v3/users/3a78167420fa4daf8856ebfc091ebf70` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:19.952741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a78167420fa4daf8856ebfc091ebf70)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:19.954636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:19.999256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a78167420fa4daf8856ebfc091ebf70'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:19.999902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a78167420fa4daf8856ebfc091ebf70'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.000768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a78167420fa4daf8856ebfc091ebf70'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.014684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a78167420fa4daf8856ebfc091ebf70'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.021525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-621639c5-597d-44a7-b14b-8dc9f90d99e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a78167420fa4daf8856ebfc091ebf70'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.034026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1003/2008] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:19 2022] DELETE /identity/v3/users/3a78167420fa4daf8856ebfc091ebf70 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:20.311445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:20.312917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:20.328242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaae14e-1d85-4ab8-9184-441a387f6ce9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:20.383983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:20.390552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:20.391233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.391577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.391863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] PATH_INFO: `/v3/projects/843afdacd2f1482f9a0f283a84720f45` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.393060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=843afdacd2f1482f9a0f283a84720f45)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:20.395123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:20.446565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843afdacd2f1482f9a0f283a84720f45', 'request_id': 'req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.454093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843afdacd2f1482f9a0f283a84720f45', 'request_id': 'req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.462277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843afdacd2f1482f9a0f283a84720f45', 'request_id': 'req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.472823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] The token cache is being invalidate because project 843afdacd2f1482f9a0f283a84720f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:20.473139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.473781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0cb8fc11-a4ed-4e52-ac69-89a7192b5364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:20.476218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1006/2009] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:20 2022] DELETE /identity/v3/projects/843afdacd2f1482f9a0f283a84720f45 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:20.480939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f86d-8dc6-442d-9e7b-d1420db84946 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.481437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f86d-8dc6-442d-9e7b-d1420db84946 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.481555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f86d-8dc6-442d-9e7b-d1420db84946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.501750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1fe4f86d-8dc6-442d-9e7b-d1420db84946 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:20.502993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1fe4f86d-8dc6-442d-9e7b-d1420db84946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:20.564358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1004/2010] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:20.572441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832d67a-0814-4ef5-a6a7-f6367f9978b3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.572849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832d67a-0814-4ef5-a6a7-f6367f9978b3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.573013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832d67a-0814-4ef5-a6a7-f6367f9978b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.601448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7832d67a-0814-4ef5-a6a7-f6367f9978b3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:20.603003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7832d67a-0814-4ef5-a6a7-f6367f9978b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:20.611033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1007/2011] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:20.615086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f86d-8dc6-442d-9e7b-d1420db84946 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:20.620189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:20.626441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f86d-8dc6-442d-9e7b-d1420db84946 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:20.632717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:20.633331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.633613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.633886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.634570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:20.634922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:20.636455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:20.648671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1005/2012] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:31:20.653022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832d67a-0814-4ef5-a6a7-f6367f9978b3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:20.663173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832d67a-0814-4ef5-a6a7-f6367f9978b3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:20.668301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:20.668930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.669196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.669691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.670202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:20.672142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:20.702204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1008/2013] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:20 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:20.706492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:20.718581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bdfae-3f11-46ed-a3d5-ae29fe28ce08 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:20.838480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:20.839166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.839534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.839840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.840570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:20.844506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:20.872730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1006/2014] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:20 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:31:20.876220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:20.890084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382da1f5-3d81-47f7-92af-7aea0efdc815 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:20.906493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:20.907292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.907676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.908240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.909115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:20.911016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:20.927322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1009/2015] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:20.931806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:20.943458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0033f0-4ccf-4d6a-b112-e87a957308f5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:20.950052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240408cc-6d0d-435b-aebe-d80516d85530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:20.950723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240408cc-6d0d-435b-aebe-d80516d85530 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:20.951722 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240408cc-6d0d-435b-aebe-d80516d85530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:20.952462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240408cc-6d0d-435b-aebe-d80516d85530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:20.952710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240408cc-6d0d-435b-aebe-d80516d85530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:20.954488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240408cc-6d0d-435b-aebe-d80516d85530 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:20.981956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1007/2016] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:20.985833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:20.996035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76caaa-5d37-4581-b411-6d9947e71793 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.000919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.001694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.001994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.002283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] PATH_INFO: `/v3/projects/355a6c9f53634ad4abb2dbc31901b368/users/7d6ab64426814adb8ffecf9b3b015707/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.018006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] RBAC: Authorizing `identity:create_grant(project_id=355a6c9f53634ad4abb2dbc31901b368, user_id=7d6ab64426814adb8ffecf9b3b015707, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.020710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.033860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1010/2017] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:20 2022] PUT /identity/v3/projects/355a6c9f53634ad4abb2dbc31901b368/users/7d6ab64426814adb8ffecf9b3b015707/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:21.044278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c30f-fe21-49dd-aed3-bd05e1dc34b1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.044550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c30f-fe21-49dd-aed3-bd05e1dc34b1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.044824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c30f-fe21-49dd-aed3-bd05e1dc34b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.072997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-20d8c30f-fe21-49dd-aed3-bd05e1dc34b1 None None] MFA Rules not processed for user `7d6ab64426814adb8ffecf9b3b015707`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:21.074217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-20d8c30f-fe21-49dd-aed3-bd05e1dc34b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:21.136460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1008/2018] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:21.140835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:21.203232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc50b3-e9b9-454e-b3c0-b4618abb312e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.207940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.208587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.208865 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.209138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.209848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.212025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.243349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1011/2019] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:21 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:21.247531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8c30f-fe21-49dd-aed3-bd05e1dc34b1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:21.258823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8c30f-fe21-49dd-aed3-bd05e1dc34b1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.308584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:21.315587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.316426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.316713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.316986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.317638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.320299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.340695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1009/2020] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:21 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:31:21.344739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:21.355341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ee961-ea22-41f0-bfc1-2cad5f7270b3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.360219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.361033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.361391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.361624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.362314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.363958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.366767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:21.377389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1012/2021] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:21.381428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:21.392126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1b494-1525-42e8-b543-5295a9d5d5da admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.396742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.397398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.397680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.397961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] PATH_INFO: `/v3/projects/b250c66e429349858c4ff1d297001561/users/bb4c452e9f2147b58a26892af7558599/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.415669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b250c66e429349858c4ff1d297001561, user_id=bb4c452e9f2147b58a26892af7558599, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.418510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.432463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1010/2022] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:21 2022] PUT /identity/v3/projects/b250c66e429349858c4ff1d297001561/users/bb4c452e9f2147b58a26892af7558599/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:21.436364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:21.447001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca3d90-f79c-4261-a959-bae6f47a7e96 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.507300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973c635-3314-48b3-a2ad-60bbbbd33505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.507849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c635-3314-48b3-a2ad-60bbbbd33505 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.508133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c635-3314-48b3-a2ad-60bbbbd33505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.508411 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973c635-3314-48b3-a2ad-60bbbbd33505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.509107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973c635-3314-48b3-a2ad-60bbbbd33505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.510702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973c635-3314-48b3-a2ad-60bbbbd33505 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.527529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1013/2023] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:31:21.529088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:21.540139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b62e03-f8fb-4187-888d-f30d19d2d023 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.544826 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.545485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.545765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.546041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] PATH_INFO: `/v3/projects/b250c66e429349858c4ff1d297001561/users/bb4c452e9f2147b58a26892af7558599/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.548006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b250c66e429349858c4ff1d297001561, user_id=bb4c452e9f2147b58a26892af7558599, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.550904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.562423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1011/2024] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:21 2022] PUT /identity/v3/projects/b250c66e429349858c4ff1d297001561/users/bb4c452e9f2147b58a26892af7558599/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:21.573407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.573698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.573976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.601250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] MFA Rules not processed for user `bb4c452e9f2147b58a26892af7558599`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:21.602454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:21.671923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1014/2025] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:21.683827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:21.685604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:21.746725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13429d65-3bca-4bd2-a06e-7d76724ba9ef admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:21.750545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d51493-cd19-4b71-b271-a5db371489af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:21.751091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d51493-cd19-4b71-b271-a5db371489af service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:21.751373 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d51493-cd19-4b71-b271-a5db371489af service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:21.751647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d51493-cd19-4b71-b271-a5db371489af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:21.812529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d51493-cd19-4b71-b271-a5db371489af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:21.813859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d51493-cd19-4b71-b271-a5db371489af service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:21.822656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1012/2026] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:22.049235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:22.050735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:22.136885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcef8a-c898-4783-85c9-e30689e97b31 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:22.145155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:22.145822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.146103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.146380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] PATH_INFO: `/v3/users/b8a8bb4f25dd41a78d3a5a196aa1e794` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.151076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8a8bb4f25dd41a78d3a5a196aa1e794)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:22.151076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:22.173420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:22.207033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8a8bb4f25dd41a78d3a5a196aa1e794'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.215023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8a8bb4f25dd41a78d3a5a196aa1e794'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.215825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8a8bb4f25dd41a78d3a5a196aa1e794'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.216505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8a8bb4f25dd41a78d3a5a196aa1e794'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.223794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8a8bb4f25dd41a78d3a5a196aa1e794'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.241302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1015/2027] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:22 2022] DELETE /identity/v3/users/b8a8bb4f25dd41a78d3a5a196aa1e794 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:22.245934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.246220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.246491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.265772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:22.267288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:22.324810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1013/2028] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:22.334291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:22.336374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:22.391626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc660b-9301-49c8-a98a-bf29e43bdfbe admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:22.395389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb0e6a-61c4-403d-a903-cd2dc6ada44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:22.396027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb0e6a-61c4-403d-a903-cd2dc6ada44c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.396308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb0e6a-61c4-403d-a903-cd2dc6ada44c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.396578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb0e6a-61c4-403d-a903-cd2dc6ada44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.406070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb0e6a-61c4-403d-a903-cd2dc6ada44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:22.407538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb0e6a-61c4-403d-a903-cd2dc6ada44c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:22.416828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1016/2029] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:22.693639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:22.695655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:22.706025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:22.710074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fad03-a935-4f27-8934-346a8392a3a2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:22.715843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:22.716495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.716776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.717116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] PATH_INFO: `/v3/projects/fbd58199ac5c444795b3fed254d2efd7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.718241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd58199ac5c444795b3fed254d2efd7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:22.720112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:22.771109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd58199ac5c444795b3fed254d2efd7', 'request_id': 'req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.777639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd58199ac5c444795b3fed254d2efd7', 'request_id': 'req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.783564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd58199ac5c444795b3fed254d2efd7', 'request_id': 'req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.798146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] The token cache is being invalidate because project fbd58199ac5c444795b3fed254d2efd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:22.798437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.799116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c6360c2-0b97-4c6f-ab57-2605a4908ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:22.801198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1014/2030] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:22 2022] DELETE /identity/v3/projects/fbd58199ac5c444795b3fed254d2efd7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:22.807978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef23df-de6e-4362-8399-2aff6272aab8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.808504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef23df-de6e-4362-8399-2aff6272aab8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.809015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef23df-de6e-4362-8399-2aff6272aab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.831117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e0ef23df-de6e-4362-8399-2aff6272aab8 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:22.832250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e0ef23df-de6e-4362-8399-2aff6272aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:22.900809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1017/2031] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:22.909564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4f157-6842-4396-abaa-7fd388aaf9e5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.910020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4f157-6842-4396-abaa-7fd388aaf9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.910400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4f157-6842-4396-abaa-7fd388aaf9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.932252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8ae4f157-6842-4396-abaa-7fd388aaf9e5 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:22.933447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8ae4f157-6842-4396-abaa-7fd388aaf9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:22.941308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1015/2032] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:22.946110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef23df-de6e-4362-8399-2aff6272aab8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:22.955993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef23df-de6e-4362-8399-2aff6272aab8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:22.961182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:22.961788 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.962070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.962345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.963072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:22.963376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:22.964963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:22.976240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1018/2033] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:31:22.980459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4f157-6842-4396-abaa-7fd388aaf9e5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:22.989529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4f157-6842-4396-abaa-7fd388aaf9e5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:22.994114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:22.994711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:22.995019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:22.995294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:22.995957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:22.997819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.024779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1016/2034] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:22 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:31:23.028996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:23.039128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575be3e2-8eb9-4f5a-89ef-a0c77692e657 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:23.064726 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:23.099345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:23.099893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.100238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.100516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.101193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:23.103314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.126949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1019/2035] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:23 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:23.131101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:23.142740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb415-d4f1-46df-bd40-8eab2801875c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:23.148070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:23.148697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.148983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.149261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.149962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:23.151617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.162140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1017/2036] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:23.166164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:23.176152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36627cd-7c17-4b52-8deb-329379244064 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:23.183539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15392ff-7190-4f5e-b384-0ef8df9cf3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:23.184257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15392ff-7190-4f5e-b384-0ef8df9cf3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.184605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15392ff-7190-4f5e-b384-0ef8df9cf3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.184971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15392ff-7190-4f5e-b384-0ef8df9cf3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.185671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15392ff-7190-4f5e-b384-0ef8df9cf3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:23.187355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15392ff-7190-4f5e-b384-0ef8df9cf3ab admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.201781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1020/2037] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:23.202755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:23.216633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edcff0-29b5-4cd8-9fdb-159ed923c2c3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:23.223347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:23.224279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.224598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.224880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] PATH_INFO: `/v3/projects/b6ef3d95467e4d27887b8ca485146986/users/d6d8cb1a751a424a8861cfd744567233/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.251016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ef3d95467e4d27887b8ca485146986, user_id=d6d8cb1a751a424a8861cfd744567233, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:23.263083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.278803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1018/2038] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:23 2022] PUT /identity/v3/projects/b6ef3d95467e4d27887b8ca485146986/users/d6d8cb1a751a424a8861cfd744567233/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:23.289312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011ee51-3a02-40c4-9273-5aa4d63ff379 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.289608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011ee51-3a02-40c4-9273-5aa4d63ff379 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.291277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011ee51-3a02-40c4-9273-5aa4d63ff379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.327154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3011ee51-3a02-40c4-9273-5aa4d63ff379 None None] MFA Rules not processed for user `d6d8cb1a751a424a8861cfd744567233`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:23.328611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3011ee51-3a02-40c4-9273-5aa4d63ff379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:23.424905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1021/2039] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:23.441141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:23.443840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:23.518348 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29b61-cca2-4e0c-810c-6cc92b0c0785 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:23.524631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d3ff-0452-4ec3-a118-8561b70de5a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:23.525273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d3ff-0452-4ec3-a118-8561b70de5a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.525556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d3ff-0452-4ec3-a118-8561b70de5a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.525831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d3ff-0452-4ec3-a118-8561b70de5a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.535808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d3ff-0452-4ec3-a118-8561b70de5a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:23.537235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d3ff-0452-4ec3-a118-8561b70de5a9 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.547134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1019/2040] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:23.575147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011ee51-3a02-40c4-9273-5aa4d63ff379 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:23.663397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011ee51-3a02-40c4-9273-5aa4d63ff379 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:23.669165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:23.669805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.670006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.670280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] PATH_INFO: `/v3/users/d6d8cb1a751a424a8861cfd744567233` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.671493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d8cb1a751a424a8861cfd744567233)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:23.673398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.717897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:23.726161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d8cb1a751a424a8861cfd744567233'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:23.733215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d8cb1a751a424a8861cfd744567233'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:23.733882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d8cb1a751a424a8861cfd744567233'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:23.734483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d8cb1a751a424a8861cfd744567233'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:23.741653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d8cb1a751a424a8861cfd744567233'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:23.759271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1022/2041] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:23 2022] DELETE /identity/v3/users/d6d8cb1a751a424a8861cfd744567233 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:23.764066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fe603-0b95-41ad-9054-e6079735384c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.764492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fe603-0b95-41ad-9054-e6079735384c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.764616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fe603-0b95-41ad-9054-e6079735384c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.782965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-180fe603-0b95-41ad-9054-e6079735384c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:23.784141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-180fe603-0b95-41ad-9054-e6079735384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:23.805174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:23.845244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1020/2042] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:23.855650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:23.856975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:23.911982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a88e3-73ba-4d46-9fad-55993a3ac680 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:23.916070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0586e3-2f53-4407-a7ae-4f1325020080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:23.916685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0586e3-2f53-4407-a7ae-4f1325020080 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:23.916960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0586e3-2f53-4407-a7ae-4f1325020080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:23.917231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0586e3-2f53-4407-a7ae-4f1325020080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:23.926860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0586e3-2f53-4407-a7ae-4f1325020080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:23.930745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0586e3-2f53-4407-a7ae-4f1325020080 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:23.940220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1023/2043] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:31:24.461504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fe603-0b95-41ad-9054-e6079735384c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:24.463375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-180fe603-0b95-41ad-9054-e6079735384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:24.481351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fe603-0b95-41ad-9054-e6079735384c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:24.490906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:24.491658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:24.492046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:24.492389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] PATH_INFO: `/v3/projects/b6ef3d95467e4d27887b8ca485146986` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:24.493787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ef3d95467e4d27887b8ca485146986)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:24.495828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:24.548087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ef3d95467e4d27887b8ca485146986', 'request_id': 'req-42d23c92-caf1-4a60-9254-01104f9a32c6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:24.554642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ef3d95467e4d27887b8ca485146986', 'request_id': 'req-42d23c92-caf1-4a60-9254-01104f9a32c6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:24.560678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ef3d95467e4d27887b8ca485146986', 'request_id': 'req-42d23c92-caf1-4a60-9254-01104f9a32c6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:24.574422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] The token cache is being invalidate because project b6ef3d95467e4d27887b8ca485146986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:24.574711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:24.575898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-42d23c92-caf1-4a60-9254-01104f9a32c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:24.577155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1021/2044] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:24 2022] DELETE /identity/v3/projects/b6ef3d95467e4d27887b8ca485146986 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:24.582564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1ebe8-c7a3-4d9a-82a4-5a14f2708aee None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:24.583088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1ebe8-c7a3-4d9a-82a4-5a14f2708aee None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:24.583214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1ebe8-c7a3-4d9a-82a4-5a14f2708aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:24.603355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2ac1ebe8-c7a3-4d9a-82a4-5a14f2708aee None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:24.604559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2ac1ebe8-c7a3-4d9a-82a4-5a14f2708aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:24.666770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1024/2045] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:24.674429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5315d-316c-4707-898b-bcd04621b924 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:24.674724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5315d-316c-4707-898b-bcd04621b924 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:24.675447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5315d-316c-4707-898b-bcd04621b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:24.705485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a2b5315d-316c-4707-898b-bcd04621b924 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:24.706805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a2b5315d-316c-4707-898b-bcd04621b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:24.725371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1022/2046] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:24.725484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1ebe8-c7a3-4d9a-82a4-5a14f2708aee None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:24.730641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1ebe8-c7a3-4d9a-82a4-5a14f2708aee None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:24.735379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:24.736001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:24.736283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:24.736556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:24.737257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:24.737562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:24.739172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:24.751588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1025/2047] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:31:24.756046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5315d-316c-4707-898b-bcd04621b924 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:24.768770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5315d-316c-4707-898b-bcd04621b924 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:24.777343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:24.778436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:24.778987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:24.779780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:24.781148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:24.783700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:24.816388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1023/2048] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:24 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:24.821002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:24.835340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97efdde-6c04-4d02-af8c-c26222aa605c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:24.901856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:24.918542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:24.919370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:24.919693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:24.919930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:24.920596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:24.922515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:24.947911 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1026/2049] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:24 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:24.952151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:24.964025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12782fc-1c48-476c-a928-39dc3972952d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:24.969342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:24.970032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:24.970324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:24.970601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:24.971335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:24.972968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:24.983834 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1024/2050] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:24.990120 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:25.003095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd179a7-91e7-4de3-9345-f44594f7d59a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:25.010142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b4dcf-5130-42e3-880e-c13dfc4b36a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:25.010333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b4dcf-5130-42e3-880e-c13dfc4b36a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:25.010593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b4dcf-5130-42e3-880e-c13dfc4b36a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:25.010802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b4dcf-5130-42e3-880e-c13dfc4b36a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:25.011603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b4dcf-5130-42e3-880e-c13dfc4b36a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:25.013370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b4dcf-5130-42e3-880e-c13dfc4b36a2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:25.034972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1027/2051] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:25.038984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:25.051661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ebc64-af08-4d2b-8f1c-7699d64fbde3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:25.057092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:25.058057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:25.058493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:25.058950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] PATH_INFO: `/v3/projects/ca1a30d14e9a4c0ebd2fd8549a09ee44/users/f96728f2ae5d4885858560396971a1b4/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:25.079395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1a30d14e9a4c0ebd2fd8549a09ee44, user_id=f96728f2ae5d4885858560396971a1b4, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:25.082845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:25.105587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1025/2052] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:25 2022] PUT /identity/v3/projects/ca1a30d14e9a4c0ebd2fd8549a09ee44/users/f96728f2ae5d4885858560396971a1b4/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:25.112030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:25.112289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:25.112568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:25.140298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:25.141595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:25.200581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1028/2053] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:25.210835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:25.212527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:25.262310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:25.267638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463c48-8405-41b9-913a-d25a4325896d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:25.272921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fa940-d7ee-4ee9-8550-3849a8c7826b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:25.273593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fa940-d7ee-4ee9-8550-3849a8c7826b service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:25.274095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fa940-d7ee-4ee9-8550-3849a8c7826b service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:25.274541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fa940-d7ee-4ee9-8550-3849a8c7826b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:25.283497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fa940-d7ee-4ee9-8550-3849a8c7826b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:25.284894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fa940-d7ee-4ee9-8550-3849a8c7826b service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:25.294050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1026/2054] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:26.221272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:26.222755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:26.293328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddb68c-6ba6-44dc-991e-f57a7710fee8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:26.298448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:26.299297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:26.299730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:26.300222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:26.302236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:26.380072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:26.381706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:26.392588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1029/2055] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:31.077186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb567-6d9b-4905-8012-ca7469f3744f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.077790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb567-6d9b-4905-8012-ca7469f3744f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.077790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb567-6d9b-4905-8012-ca7469f3744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.110457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-98efb567-6d9b-4905-8012-ca7469f3744f None None] MFA Rules not processed for user `f96728f2ae5d4885858560396971a1b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:31.111686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-98efb567-6d9b-4905-8012-ca7469f3744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:31.187456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1027/2056] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:31.192075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.203633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07287e1-e787-4b5a-b58b-52e0e319666c service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.209040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.209666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.209950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.210224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.210908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.212834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.230267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:31.247390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1030/2057] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:31 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:31.251576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb567-6d9b-4905-8012-ca7469f3744f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.262815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb567-6d9b-4905-8012-ca7469f3744f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.339440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.340136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.340480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.340835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.341496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.343437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.367146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1028/2058] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:31 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:31.370792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.381696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b92ad-38eb-406f-b622-1c10b417ad7b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.388537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.389517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.389799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.390073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.390797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.392652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.406057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1031/2059] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:31:31.410439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.421616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1483295-7ad0-4a83-b1fc-6eb19826befe admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.427547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.429101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.429101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.429447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] PATH_INFO: `/v3/projects/b9dc80dd9a274082a05cba603b5def7c/users/0b066818cbfe40df85381d31a947a1d1/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.448940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dc80dd9a274082a05cba603b5def7c, user_id=0b066818cbfe40df85381d31a947a1d1, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.451779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.468589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1029/2060] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:31 2022] PUT /identity/v3/projects/b9dc80dd9a274082a05cba603b5def7c/users/0b066818cbfe40df85381d31a947a1d1/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:31.472791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.483991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7faa6-5429-4c60-9201-f5ff51555ea7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.551378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.556327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:31.557813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.558769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.559432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.559541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.560446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.562239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.577941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d9dd7-6862-45c0-9381-9b2db7312b7f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.579905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1032/2061] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:31.580840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.587437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.590245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.590773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.591401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] PATH_INFO: `/v3/users/7d6ab64426814adb8ffecf9b3b015707` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.592125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c3a06-80dd-4c71-a063-a285c7108da6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.594073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6ab64426814adb8ffecf9b3b015707)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.598042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.601356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c665-a06e-42fd-9fde-8a7d3e5ab3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.602171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c665-a06e-42fd-9fde-8a7d3e5ab3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.602452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c665-a06e-42fd-9fde-8a7d3e5ab3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.602748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c665-a06e-42fd-9fde-8a7d3e5ab3e7 admin admin] PATH_INFO: `/v3/projects/b9dc80dd9a274082a05cba603b5def7c/users/0b066818cbfe40df85381d31a947a1d1/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.604845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c665-a06e-42fd-9fde-8a7d3e5ab3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dc80dd9a274082a05cba603b5def7c, user_id=0b066818cbfe40df85381d31a947a1d1, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.607687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c665-a06e-42fd-9fde-8a7d3e5ab3e7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.623919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1033/2062] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:31 2022] PUT /identity/v3/projects/b9dc80dd9a274082a05cba603b5def7c/users/0b066818cbfe40df85381d31a947a1d1/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:31.666947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:31.676853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6ab64426814adb8ffecf9b3b015707'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:31.677863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6ab64426814adb8ffecf9b3b015707'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:31.680301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6ab64426814adb8ffecf9b3b015707'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:31.699086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6ab64426814adb8ffecf9b3b015707'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:31.707399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6ab64426814adb8ffecf9b3b015707'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:31.723107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1030/2063] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:31 2022] DELETE /identity/v3/users/7d6ab64426814adb8ffecf9b3b015707 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:31.727544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.728155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.728489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.753634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:31.754921 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:31.831034 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1034/2064] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:31.841670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:31.843782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:31.919696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69b2d0-851e-46ed-ad06-75963609ffea admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:31.924357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:31.925029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:31.925314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:31.925586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:31.938842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:31.940339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:31.953049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1031/2065] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:32.414011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:32.416088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:32.431544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700e42-0ba1-4279-a19d-a71646cc0f8d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:32.440149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:32.440852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:32.441134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:32.441409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] PATH_INFO: `/v3/projects/355a6c9f53634ad4abb2dbc31901b368` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:32.442908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=355a6c9f53634ad4abb2dbc31901b368)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:32.445202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:32.503688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '355a6c9f53634ad4abb2dbc31901b368', 'request_id': 'req-39efcf28-4995-4d9e-be41-13f672e06f9d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.514255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '355a6c9f53634ad4abb2dbc31901b368', 'request_id': 'req-39efcf28-4995-4d9e-be41-13f672e06f9d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.524780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '355a6c9f53634ad4abb2dbc31901b368', 'request_id': 'req-39efcf28-4995-4d9e-be41-13f672e06f9d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.526395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:32.548602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] The token cache is being invalidate because project 355a6c9f53634ad4abb2dbc31901b368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:32.549304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.550317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.552630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1035/2066] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:32 2022] DELETE /identity/v3/projects/355a6c9f53634ad4abb2dbc31901b368 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:32.557686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:32.575556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158867c2-96ec-4f49-897a-c4e7763403e1 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:32.665138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:32.665648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:32.665930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:32.666198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] PATH_INFO: `/v3/users/bb4c452e9f2147b58a26892af7558599` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:32.667451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4c452e9f2147b58a26892af7558599)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:32.669712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:32.758307 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4c452e9f2147b58a26892af7558599'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.760431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4c452e9f2147b58a26892af7558599'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.761100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4c452e9f2147b58a26892af7558599'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.784853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:32.789225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4c452e9f2147b58a26892af7558599'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.804206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f6624711-cce7-496f-b1b3-b7f7d64f1225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4c452e9f2147b58a26892af7558599'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:32.845631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1032/2067] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:32 2022] DELETE /identity/v3/users/bb4c452e9f2147b58a26892af7558599 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:33.233980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:33.236319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:33.255106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efcf28-4995-4d9e-be41-13f672e06f9d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:33.333149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:33.334805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:33.335283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:33.336212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] PATH_INFO: `/v3/projects/b250c66e429349858c4ff1d297001561` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:33.338222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b250c66e429349858c4ff1d297001561)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:33.340845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:33.418263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b250c66e429349858c4ff1d297001561', 'request_id': 'req-13a115aa-419a-415c-b532-5faa387d708a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:33.425553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b250c66e429349858c4ff1d297001561', 'request_id': 'req-13a115aa-419a-415c-b532-5faa387d708a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:33.435406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b250c66e429349858c4ff1d297001561', 'request_id': 'req-13a115aa-419a-415c-b532-5faa387d708a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:33.447210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] The token cache is being invalidate because project b250c66e429349858c4ff1d297001561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:33.447833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:33.449810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-13a115aa-419a-415c-b532-5faa387d708a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:33.452503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1036/2068] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:33 2022] DELETE /identity/v3/projects/b250c66e429349858c4ff1d297001561 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:33.460866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417065b-cb95-4f34-b009-5eaae70f3704 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:33.461446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417065b-cb95-4f34-b009-5eaae70f3704 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:33.462084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417065b-cb95-4f34-b009-5eaae70f3704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:33.494233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c417065b-cb95-4f34-b009-5eaae70f3704 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:33.500522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c417065b-cb95-4f34-b009-5eaae70f3704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:33.637468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14766d27-1962-4be7-ae5f-fac12ee84d45 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:33.637468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14766d27-1962-4be7-ae5f-fac12ee84d45 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:33.637468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14766d27-1962-4be7-ae5f-fac12ee84d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:33.649460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1033/2069] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:33.678072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-14766d27-1962-4be7-ae5f-fac12ee84d45 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:33.680036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-14766d27-1962-4be7-ae5f-fac12ee84d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:33.690730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1037/2070] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:33.695181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417065b-cb95-4f34-b009-5eaae70f3704 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:33.726564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417065b-cb95-4f34-b009-5eaae70f3704 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:33.758282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:33.759265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:33.759770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:33.760262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:33.764998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:33.766543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:33.769584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:33.803995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1034/2071] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:31:33.814002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14766d27-1962-4be7-ae5f-fac12ee84d45 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:33.830775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14766d27-1962-4be7-ae5f-fac12ee84d45 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:33.860403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:33.861198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:33.861481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:33.861758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:33.862450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:33.865132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:33.977253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1038/2072] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:33 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:33.981002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:34.015848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed4ac7-a2a0-444f-acc8-300d9d96c5ad admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:34.146157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:34.147861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:34.147861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:34.148236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:34.149129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:34.151577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:34.186605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1035/2073] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:33 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:34.186824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:34.211887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102cd03-da2e-468a-b0cb-c69debe3ccff admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:34.223209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:34.224720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:34.226858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:34.228281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:34.229243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:34.237533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:34.273569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1039/2074] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:34.276012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:34.288068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a6934-fcbc-4b3b-a1e1-dd397f222801 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:34.294074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c88963-3dc0-4576-a849-1a246fd40b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:34.295027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88963-3dc0-4576-a849-1a246fd40b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:34.295691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88963-3dc0-4576-a849-1a246fd40b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:34.296296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88963-3dc0-4576-a849-1a246fd40b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:34.297345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c88963-3dc0-4576-a849-1a246fd40b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:34.299245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c88963-3dc0-4576-a849-1a246fd40b4a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:34.327960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1036/2075] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:31:34.329433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:34.347204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd3b44-3a9f-4913-8cfa-6d87af1e7a5f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:34.359351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:34.359351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:34.359351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:34.359351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] PATH_INFO: `/v3/projects/59e7a9fef10f4aba994347a5d714f17f/users/eb6f411ed91946eba54e46799edb12d3/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:34.404430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e7a9fef10f4aba994347a5d714f17f, user_id=eb6f411ed91946eba54e46799edb12d3, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:34.422701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:34.462140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1040/2076] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:34 2022] PUT /identity/v3/projects/59e7a9fef10f4aba994347a5d714f17f/users/eb6f411ed91946eba54e46799edb12d3/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:31:34.464580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eca79-ed8b-49f1-bb6b-59a9da227947 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:34.464793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eca79-ed8b-49f1-bb6b-59a9da227947 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:34.465154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eca79-ed8b-49f1-bb6b-59a9da227947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:34.513869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-bf3eca79-ed8b-49f1-bb6b-59a9da227947 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:34.518287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bf3eca79-ed8b-49f1-bb6b-59a9da227947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:34.617552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:34.680276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1037/2077] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:34.707765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:34.709649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:34.751110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:34.834187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df92ef-0a63-4539-8903-e73e00467411 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:34.838619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:34.839395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:34.839842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:34.840195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:34.859868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:34.861368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:34.874215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1041/2078] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:35.795079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb88ab6-10fa-4a1c-baac-63882fff149c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:35.799845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb88ab6-10fa-4a1c-baac-63882fff149c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:35.800571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb88ab6-10fa-4a1c-baac-63882fff149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:35.983814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9eb88ab6-10fa-4a1c-baac-63882fff149c None None] MFA Rules not processed for user `eb6f411ed91946eba54e46799edb12d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:35.985103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9eb88ab6-10fa-4a1c-baac-63882fff149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:36.144515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1038/2079] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:36.155568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:36.158145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:36.250575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c0a69-adc0-43c2-b2c1-03c4d6610553 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:36.257310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:36.258438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:36.259165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:36.259621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:36.270179 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:36.271884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:36.287482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1042/2080] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:37.209812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d613ff-debe-482b-ac73-e2c185365ee7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:37.210355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d613ff-debe-482b-ac73-e2c185365ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:37.210652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d613ff-debe-482b-ac73-e2c185365ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:37.249965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-33d613ff-debe-482b-ac73-e2c185365ee7 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:37.251543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-33d613ff-debe-482b-ac73-e2c185365ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:37.402711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1039/2081] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:37.467868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:37.469903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:37.601386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332fda-e443-49f4-a9e4-457453825b20 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:37.607416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a996ed-a196-4e93-b8fa-c9160327cbd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:37.608310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a996ed-a196-4e93-b8fa-c9160327cbd9 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:37.608987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a996ed-a196-4e93-b8fa-c9160327cbd9 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:37.609492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a996ed-a196-4e93-b8fa-c9160327cbd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:37.635297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a996ed-a196-4e93-b8fa-c9160327cbd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:37.637444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a996ed-a196-4e93-b8fa-c9160327cbd9 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:37.657177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1043/2082] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:31:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:31:38.141161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ae31c-d7f4-449f-a5e7-1c4f87a9ff5b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:38.141881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ae31c-d7f4-449f-a5e7-1c4f87a9ff5b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:38.142418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ae31c-d7f4-449f-a5e7-1c4f87a9ff5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:38.168063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1a3ae31c-d7f4-449f-a5e7-1c4f87a9ff5b None None] MFA Rules not processed for user `eb6f411ed91946eba54e46799edb12d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:38.169976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1a3ae31c-d7f4-449f-a5e7-1c4f87a9ff5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:38.192590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1040/2083] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:38.819960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0255c76-8ce3-486e-ab72-2f567467890c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:38.820910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0255c76-8ce3-486e-ab72-2f567467890c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:38.821374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0255c76-8ce3-486e-ab72-2f567467890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:38.864231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b0255c76-8ce3-486e-ab72-2f567467890c None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:38.866254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b0255c76-8ce3-486e-ab72-2f567467890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:38.880868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c2eda-28b7-4c19-8f5e-ef322b69c540 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:38.881792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c2eda-28b7-4c19-8f5e-ef322b69c540 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:38.882256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c2eda-28b7-4c19-8f5e-ef322b69c540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:38.896066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1044/2084] 10.222.0.17 () {58 vars in 967 bytes} [Sat Aug 6 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:38.918812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d76c2eda-28b7-4c19-8f5e-ef322b69c540 None None] MFA Rules not processed for user `0b066818cbfe40df85381d31a947a1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:38.920046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0255c76-8ce3-486e-ab72-2f567467890c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:38.920631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d76c2eda-28b7-4c19-8f5e-ef322b69c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:38.935735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0255c76-8ce3-486e-ab72-2f567467890c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:38.946905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:38.947630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:38.947960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:38.948240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:38.966762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:38.968397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:38.988898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1045/2085] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:31:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 14:31:38.997836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:39.028202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1041/2086] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:39.036376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:39.039057 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:39.122533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:39.175311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeac66-030b-428b-a3ff-5a821e06b8ef service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:39.181274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:39.181884 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:39.182119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:39.182823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:39.184955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:39.286771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:39.288983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:39.329068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1046/2087] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:39.476235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f7fb-abdc-404c-b1e3-10a370342f07 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:39.476235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f7fb-abdc-404c-b1e3-10a370342f07 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:39.476825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f7fb-abdc-404c-b1e3-10a370342f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:39.524033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4742f7fb-abdc-404c-b1e3-10a370342f07 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:39.525261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4742f7fb-abdc-404c-b1e3-10a370342f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:39.546258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1042/2088] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:39.555748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:39.573771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e0ac-924e-4424-b8d2-acb57fbce0b9 service glance] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:39.586842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:39.587616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:39.587933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:39.588207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:39.593191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd2d26-dac9-4044-be06-ffc2a50302da None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:39.593988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd2d26-dac9-4044-be06-ffc2a50302da None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:39.594157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd2d26-dac9-4044-be06-ffc2a50302da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:39.615458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:39.617236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:39.619536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-89bd2d26-dac9-4044-be06-ffc2a50302da None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:39.620779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-89bd2d26-dac9-4044-be06-ffc2a50302da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:39.630415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1043/2089] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:39.637600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1047/2090] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:31:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:31:39.674685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:39.692897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbb161-fbd4-4e71-bc1a-23f801565e1f service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:39.703292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:39.704745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:39.705228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:39.705659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:39.719823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:39.737811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:39.774325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1048/2091] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:31:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:31:40.381419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b105-70e2-4357-9fea-ef50b9b86b2b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:40.382288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b105-70e2-4357-9fea-ef50b9b86b2b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:40.382739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b105-70e2-4357-9fea-ef50b9b86b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:40.410587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9a14b105-70e2-4357-9fea-ef50b9b86b2b None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:40.411918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9a14b105-70e2-4357-9fea-ef50b9b86b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:40.420998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1044/2092] 10.222.0.17 () {58 vars in 967 bytes} [Sat Aug 6 14:31:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:40.484162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:40.505962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da99fbe-bd02-4ccd-829a-e9933df9c68b service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:40.522293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a0d8f-66ee-4e68-9f94-e66b3a2684a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:40.523440 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a0d8f-66ee-4e68-9f94-e66b3a2684a9 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:40.523959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a0d8f-66ee-4e68-9f94-e66b3a2684a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:40.525509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a0d8f-66ee-4e68-9f94-e66b3a2684a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:40.554998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a0d8f-66ee-4e68-9f94-e66b3a2684a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:40.562944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a0d8f-66ee-4e68-9f94-e66b3a2684a9 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:40.609706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1049/2093] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:31:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:31:43.574511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc31f35-c420-46ee-9dc9-40587e60ee1a None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:43.575107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc31f35-c420-46ee-9dc9-40587e60ee1a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:43.575240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc31f35-c420-46ee-9dc9-40587e60ee1a None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:43.576823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1045/2094] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:31:43.637643 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb7f2-f30f-459d-98c6-6ef7716b654b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:43.638231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb7f2-f30f-459d-98c6-6ef7716b654b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:43.638231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb7f2-f30f-459d-98c6-6ef7716b654b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:43.665965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d9dcb7f2-f30f-459d-98c6-6ef7716b654b None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:43.667842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d9dcb7f2-f30f-459d-98c6-6ef7716b654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:43.816121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1050/2095] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:43.818205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc31f35-c420-46ee-9dc9-40587e60ee1a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:43.820768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-adc31f35-c420-46ee-9dc9-40587e60ee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:43.868529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961476a-f9ce-4659-8473-56a627325913 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:43.870805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961476a-f9ce-4659-8473-56a627325913 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:43.871276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961476a-f9ce-4659-8473-56a627325913 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:43.883688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1051/2096] 10.222.0.17 () {54 vars in 961 bytes} [Sat Aug 6 14:31:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:31:43.889149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1480f-bca4-4671-a7b7-a119f4895c43 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:43.889439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1480f-bca4-4671-a7b7-a119f4895c43 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:43.889719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1480f-bca4-4671-a7b7-a119f4895c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:43.906828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc31f35-c420-46ee-9dc9-40587e60ee1a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:43.926354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:43.929278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:43.929565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:43.929843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] PATH_INFO: `/v3/users/8a1e8506e8b34a8d9c59b4acdbf23bb8` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:43.937690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1e8506e8b34a8d9c59b4acdbf23bb8)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:43.942574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:43.958290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9ad1480f-bca4-4671-a7b7-a119f4895c43 None None] MFA Rules not processed for user `d607b8bbb8f047f0bbaf8dad33d47247`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:43.961191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9ad1480f-bca4-4671-a7b7-a119f4895c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:43.985282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1052/2097] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Aug 6 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:43.999719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1480f-bca4-4671-a7b7-a119f4895c43 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:44.009978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1480f-bca4-4671-a7b7-a119f4895c43 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:44.020367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:44.021157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:44.021563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:44.021942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:44.038185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:44.040233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:44.061224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1053/2098] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:31:44.133184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1e8506e8b34a8d9c59b4acdbf23bb8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.134188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1e8506e8b34a8d9c59b4acdbf23bb8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.135940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1e8506e8b34a8d9c59b4acdbf23bb8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.160100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1e8506e8b34a8d9c59b4acdbf23bb8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.174451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9e16dacd-f9ba-46f5-b270-71447dc98743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1e8506e8b34a8d9c59b4acdbf23bb8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.211240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1046/2099] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:43 2022] DELETE /identity/v3/users/8a1e8506e8b34a8d9c59b4acdbf23bb8 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:44.235442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:44.238543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:44.270211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56acae-abb2-4d6f-8148-697d09c65aa8 service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:44.389159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:44.479639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:44.480208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:44.480474 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:44.480748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] PATH_INFO: `/v3/users/de557945688e4d9f944310015bb42163` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:44.491842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de557945688e4d9f944310015bb42163)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:44.494043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:44.597256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de557945688e4d9f944310015bb42163'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.623425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de557945688e4d9f944310015bb42163'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.624185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de557945688e4d9f944310015bb42163'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.625610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de557945688e4d9f944310015bb42163'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.644683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de557945688e4d9f944310015bb42163'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:44.694437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1054/2100] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:44 2022] DELETE /identity/v3/users/de557945688e4d9f944310015bb42163 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:44.698702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:44.699990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:44.700482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:44.763332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:44.766401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:44.916169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1047/2101] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:44.926222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:44.928015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:44.983141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:44.986452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:45.006650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f05d1-244a-4083-8401-12823b9bc73a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:45.019916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:45.020598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.021034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.021120 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] PATH_INFO: `/v3/projects/f7ead8c81ff3486683d21f671acac6e0` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.024523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ead8c81ff3486683d21f671acac6e0)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:45.027900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:45.060945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c798-0b2b-4fbf-9f26-6131405ae262 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:45.070328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f5657-89a6-40ee-b4bb-cc09876e4f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:45.070997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f5657-89a6-40ee-b4bb-cc09876e4f4d service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.071290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f5657-89a6-40ee-b4bb-cc09876e4f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.071553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f5657-89a6-40ee-b4bb-cc09876e4f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.089695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f5657-89a6-40ee-b4bb-cc09876e4f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:45.091245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f5657-89a6-40ee-b4bb-cc09876e4f4d service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:45.105151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1055/2102] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:45.156182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:45.161213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ead8c81ff3486683d21f671acac6e0', 'request_id': 'req-d4ec6cce-3f1b-437d-99f9-2da9592744f4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.175950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ead8c81ff3486683d21f671acac6e0', 'request_id': 'req-d4ec6cce-3f1b-437d-99f9-2da9592744f4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.183649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ead8c81ff3486683d21f671acac6e0', 'request_id': 'req-d4ec6cce-3f1b-437d-99f9-2da9592744f4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.209725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] The token cache is being invalidate because project f7ead8c81ff3486683d21f671acac6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:45.210426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.213542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4ec6cce-3f1b-437d-99f9-2da9592744f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.216699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1048/2103] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:44 2022] DELETE /identity/v3/projects/f7ead8c81ff3486683d21f671acac6e0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:45.223752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d41441-a043-40a6-a1d6-e3f9f2bb8def None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.224043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d41441-a043-40a6-a1d6-e3f9f2bb8def None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.224317 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d41441-a043-40a6-a1d6-e3f9f2bb8def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.250113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-89d41441-a043-40a6-a1d6-e3f9f2bb8def None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:45.251383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-89d41441-a043-40a6-a1d6-e3f9f2bb8def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:45.346591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:45.379861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433c797-05c7-46d2-b16f-006f66c29d95 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.380196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433c797-05c7-46d2-b16f-006f66c29d95 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.380414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433c797-05c7-46d2-b16f-006f66c29d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.385331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1056/2104] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:45.404637 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e433c797-05c7-46d2-b16f-006f66c29d95 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:45.405985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e433c797-05c7-46d2-b16f-006f66c29d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:45.418624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1049/2105] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:45.419700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d41441-a043-40a6-a1d6-e3f9f2bb8def None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:45.436972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d41441-a043-40a6-a1d6-e3f9f2bb8def None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:45.442632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:45.444306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.444762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.445188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.446060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:45.446656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:45.448530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:45.466555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1057/2106] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:31:45.470585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433c797-05c7-46d2-b16f-006f66c29d95 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:45.482762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433c797-05c7-46d2-b16f-006f66c29d95 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:45.483986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:45.487660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:45.496398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:45.497173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.497554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.498237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.499976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:45.502118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:45.514584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a9149-881a-484a-8aca-665582dbaa87 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:45.521139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:45.522783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.523098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.523371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] PATH_INFO: `/v3/projects/1d02ee1ad57f4d38acf0bc95ab43f614` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.524668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d02ee1ad57f4d38acf0bc95ab43f614)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:45.526745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:45.579704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1050/2107] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:45 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:45.589924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:45.628351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d02ee1ad57f4d38acf0bc95ab43f614', 'request_id': 'req-84cb603c-75e4-45f6-b4ef-630e70ad768f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.642019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d02ee1ad57f4d38acf0bc95ab43f614', 'request_id': 'req-84cb603c-75e4-45f6-b4ef-630e70ad768f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.653689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d02ee1ad57f4d38acf0bc95ab43f614', 'request_id': 'req-84cb603c-75e4-45f6-b4ef-630e70ad768f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.665743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] The token cache is being invalidate because project 1d02ee1ad57f4d38acf0bc95ab43f614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:45.666252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.667153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:45.670244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1058/2108] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:45 2022] DELETE /identity/v3/projects/1d02ee1ad57f4d38acf0bc95ab43f614 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:45.718954 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362760b-3c6d-476b-9861-21bd7badfa39 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:45.899526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:45.901410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.901779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.902129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.902930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:45.904939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:45.947350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1051/2109] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:45 2022] POST /identity/v3/users => generated 284 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:45.951842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:45.953576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:45.968757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb603c-75e4-45f6-b4ef-630e70ad768f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:45.975127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:45.975774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:45.976062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:45.976682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:45.977021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:45.978878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:45.993699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1059/2110] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:45.994978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:46.008631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96f58-b035-4874-9ce6-1460bc915fc8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:46.036820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17b1cb-480d-45b1-bb51-3c039a2f3fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:46.037674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17b1cb-480d-45b1-bb51-3c039a2f3fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:46.037966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17b1cb-480d-45b1-bb51-3c039a2f3fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:46.038392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17b1cb-480d-45b1-bb51-3c039a2f3fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:46.039195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17b1cb-480d-45b1-bb51-3c039a2f3fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:46.040936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17b1cb-480d-45b1-bb51-3c039a2f3fd3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:46.067073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1052/2111] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:46.071917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:46.087628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84eb54-ef3a-4959-9162-3e80cb76bddc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:46.093416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:46.094205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:46.094928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:46.094928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] PATH_INFO: `/v3/projects/764837d6f4904889b4f93e56399039dc/users/27605fd667be4c1da906429303334a4c/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:46.114386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764837d6f4904889b4f93e56399039dc, user_id=27605fd667be4c1da906429303334a4c, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:46.118125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:46.137928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1060/2112] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:46 2022] PUT /identity/v3/projects/764837d6f4904889b4f93e56399039dc/users/27605fd667be4c1da906429303334a4c/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:46.148525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90f8c-ef46-4e14-8d18-2b89bc6965a9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:46.148855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90f8c-ef46-4e14-8d18-2b89bc6965a9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:46.149209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90f8c-ef46-4e14-8d18-2b89bc6965a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:46.199217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b3c90f8c-ef46-4e14-8d18-2b89bc6965a9 None None] MFA Rules not processed for user `27605fd667be4c1da906429303334a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:46.200559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b3c90f8c-ef46-4e14-8d18-2b89bc6965a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:46.256971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:46.321944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1053/2113] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:31:46.335951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:46.337685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:46.430969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc69ca-f038-4b9b-81c1-9d3850821f54 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:46.435695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d6666-c396-4124-9498-868e0eabda97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:46.436609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d6666-c396-4124-9498-868e0eabda97 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:46.436806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d6666-c396-4124-9498-868e0eabda97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:46.437182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d6666-c396-4124-9498-868e0eabda97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:46.466656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d6666-c396-4124-9498-868e0eabda97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:46.468227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d6666-c396-4124-9498-868e0eabda97 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:46.493646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1061/2114] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:52.366950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:52.367543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:52.367543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:52.389718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] MFA Rules not processed for user `f96728f2ae5d4885858560396971a1b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:52.390990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:52.461230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1054/2115] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:54.588689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff118733-efd7-4544-a9e4-ab19dbc8f215 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:54.589214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff118733-efd7-4544-a9e4-ab19dbc8f215 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:54.589309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff118733-efd7-4544-a9e4-ab19dbc8f215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:54.608789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ff118733-efd7-4544-a9e4-ab19dbc8f215 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:54.610014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ff118733-efd7-4544-a9e4-ab19dbc8f215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:54.626953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:54.669547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1062/2116] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:54.682324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:54.683786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:54.754325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682ab94-db63-4dde-9203-25add35ebf41 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:54.760074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e74095-f095-443e-9c7f-a9b94279eecf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:54.760781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e74095-f095-443e-9c7f-a9b94279eecf service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:54.761143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e74095-f095-443e-9c7f-a9b94279eecf service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:54.761488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e74095-f095-443e-9c7f-a9b94279eecf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:54.770291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e74095-f095-443e-9c7f-a9b94279eecf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:54.772778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e74095-f095-443e-9c7f-a9b94279eecf service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:54.782630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1055/2117] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:31:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:31:56.161502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff118733-efd7-4544-a9e4-ab19dbc8f215 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:56.279131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff118733-efd7-4544-a9e4-ab19dbc8f215 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:56.288685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:56.289454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:56.289741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:56.290021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] PATH_INFO: `/v3/users/27605fd667be4c1da906429303334a4c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:56.291774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27605fd667be4c1da906429303334a4c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:56.294091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:56.353874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27605fd667be4c1da906429303334a4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:56.364512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27605fd667be4c1da906429303334a4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:56.365359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27605fd667be4c1da906429303334a4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:56.366133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27605fd667be4c1da906429303334a4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:56.376119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27605fd667be4c1da906429303334a4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:56.395384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1063/2118] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:56 2022] DELETE /identity/v3/users/27605fd667be4c1da906429303334a4c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:56.400695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:56.402168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:56.402699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:56.438993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:56.440246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:56.491025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:56.507777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1056/2119] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:56.522388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:56.524139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:56.596563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:56.603316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f972-7b99-43e1-aee6-9e82c30325c7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:56.608209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420e7e7-88a6-4741-8faa-f118fd676d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:56.608875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420e7e7-88a6-4741-8faa-f118fd676d28 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:56.609292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420e7e7-88a6-4741-8faa-f118fd676d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:56.609571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420e7e7-88a6-4741-8faa-f118fd676d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:56.629353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420e7e7-88a6-4741-8faa-f118fd676d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:56.631651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420e7e7-88a6-4741-8faa-f118fd676d28 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:56.641991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1064/2120] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:57.034206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.035832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:57.059843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059df54-1bd8-463f-8082-aef9bf70538a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.071577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.072854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.073437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.073995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] PATH_INFO: `/v3/projects/764837d6f4904889b4f93e56399039dc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.075880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764837d6f4904889b4f93e56399039dc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.078132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.127059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764837d6f4904889b4f93e56399039dc', 'request_id': 'req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:57.135970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764837d6f4904889b4f93e56399039dc', 'request_id': 'req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:57.142399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764837d6f4904889b4f93e56399039dc', 'request_id': 'req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:57.155192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] The token cache is being invalidate because project 764837d6f4904889b4f93e56399039dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:57.155478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:57.156340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f7774e0e-a922-49f6-bd74-9ae2bb3f2774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:57.158265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1057/2121] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:57 2022] DELETE /identity/v3/projects/764837d6f4904889b4f93e56399039dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:57.163030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c71f90-ad9e-470e-9bd7-034e77b7a5f2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.163534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c71f90-ad9e-470e-9bd7-034e77b7a5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.163772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c71f90-ad9e-470e-9bd7-034e77b7a5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.185544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e7c71f90-ad9e-470e-9bd7-034e77b7a5f2 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:57.186904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e7c71f90-ad9e-470e-9bd7-034e77b7a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:57.250030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1065/2122] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:57.259909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d856a-e511-45ee-bb5b-abb099eb1b48 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.260200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d856a-e511-45ee-bb5b-abb099eb1b48 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.260470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d856a-e511-45ee-bb5b-abb099eb1b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.283238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9d3d856a-e511-45ee-bb5b-abb099eb1b48 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:57.284523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9d3d856a-e511-45ee-bb5b-abb099eb1b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:57.292899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1058/2123] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:57.297513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c71f90-ad9e-470e-9bd7-034e77b7a5f2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.309259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c71f90-ad9e-470e-9bd7-034e77b7a5f2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.314129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.314782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.315071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.315351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.316056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:31:57.316359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.318647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.331348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1066/2124] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:31:57.335824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d856a-e511-45ee-bb5b-abb099eb1b48 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.345013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d856a-e511-45ee-bb5b-abb099eb1b48 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.349584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.350182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.350461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.350733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.351419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.353329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.382631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1059/2125] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:57 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:57.386820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.397521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55986830-4e48-4d40-aa77-436aa1c08321 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.463206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.463898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.464171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.464449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.465136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.467541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.498780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1067/2126] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:57 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:57.500144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.512310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2b2f7-8468-4b97-a25c-d7007c506145 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.517626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.518473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.518825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.519201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.520304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.522039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.536101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1060/2127] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:57.538905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.550128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2082-d064-47f3-98f1-745afc4abc68 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.558732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3caa3-58a5-48a5-918e-287da8af0187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.562007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3caa3-58a5-48a5-918e-287da8af0187 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.562475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3caa3-58a5-48a5-918e-287da8af0187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.562989 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3caa3-58a5-48a5-918e-287da8af0187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.564743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3caa3-58a5-48a5-918e-287da8af0187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.569962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3caa3-58a5-48a5-918e-287da8af0187 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.590719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1068/2128] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:57.592561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.603885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80d285-ed1b-479e-9321-31b52863db0d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.609249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.610061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.610408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.610967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] PATH_INFO: `/v3/projects/df624255d8f3455daa0de4f9bf4d8dd9/users/bfc169569e1641c980526abeb3a6f463/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.630188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df624255d8f3455daa0de4f9bf4d8dd9, user_id=bfc169569e1641c980526abeb3a6f463, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.633351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.650837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1061/2129] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:57 2022] PUT /identity/v3/projects/df624255d8f3455daa0de4f9bf4d8dd9/users/bfc169569e1641c980526abeb3a6f463/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:57.662121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924bb28-789d-4e49-81e2-12080a79f8e4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.662500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924bb28-789d-4e49-81e2-12080a79f8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.662848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924bb28-789d-4e49-81e2-12080a79f8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.691932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d924bb28-789d-4e49-81e2-12080a79f8e4 None None] MFA Rules not processed for user `bfc169569e1641c980526abeb3a6f463`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:57.693181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d924bb28-789d-4e49-81e2-12080a79f8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:57.738023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:57.785613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1069/2130] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:57.793246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:57.882914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071afdb4-6e5b-41f2-857b-49b17ac43a66 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:57.887686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:57.888317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:57.888596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:57.888869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:57.889558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:57.891488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:57.936879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:57.972369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1062/2131] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:31:57 2022] POST /identity/v3/projects => generated 405 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:57.979402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924bb28-789d-4e49-81e2-12080a79f8e4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:58.017206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924bb28-789d-4e49-81e2-12080a79f8e4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:58.138022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:58.138659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.138988 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.139338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.140073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:58.142391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:58.170110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1070/2132] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:31:57 2022] POST /identity/v3/users => generated 309 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:31:58.174263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:58.186855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ead6e-7cc0-4dcb-99d1-30532950b4b2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:58.192159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:58.193368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.193775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.194138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.194948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:58.196670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:58.208410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1063/2133] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:31:58.214493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:58.231859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410375a-d052-4b52-b7ef-400c7947c661 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:58.239585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:58.241111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.241333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.241607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] PATH_INFO: `/v3/projects/2546546493de4b078f61e29ffd4471d6/users/e1cd100457b54531a6e9fe53001198c4/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.258256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2546546493de4b078f61e29ffd4471d6, user_id=e1cd100457b54531a6e9fe53001198c4, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:58.261030 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:58.278080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1071/2134] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:58 2022] PUT /identity/v3/projects/2546546493de4b078f61e29ffd4471d6/users/e1cd100457b54531a6e9fe53001198c4/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:58.282839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:58.298649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d2a7d-b4fc-4a27-9d66-ad424c04631d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:58.357256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b907f-94dc-4a46-b841-a582b675720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:58.357878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b907f-94dc-4a46-b841-a582b675720d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.358181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b907f-94dc-4a46-b841-a582b675720d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.358466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b907f-94dc-4a46-b841-a582b675720d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.359169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b907f-94dc-4a46-b841-a582b675720d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:58.360737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b907f-94dc-4a46-b841-a582b675720d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:58.371882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1064/2135] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:31:58.375547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:58.386155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5907ec2-ac05-4b3c-8496-60ce5f56f231 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:58.390848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:58.391486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.391760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.392031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] PATH_INFO: `/v3/projects/2546546493de4b078f61e29ffd4471d6/users/e1cd100457b54531a6e9fe53001198c4/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.393944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2546546493de4b078f61e29ffd4471d6, user_id=e1cd100457b54531a6e9fe53001198c4, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:58.396685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:58.409922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1072/2136] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:31:58 2022] PUT /identity/v3/projects/2546546493de4b078f61e29ffd4471d6/users/e1cd100457b54531a6e9fe53001198c4/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:58.418951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e8ac2-7c87-4693-8631-231f6c169f0e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.419209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e8ac2-7c87-4693-8631-231f6c169f0e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.419489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e8ac2-7c87-4693-8631-231f6c169f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.450900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-375e8ac2-7c87-4693-8631-231f6c169f0e None None] MFA Rules not processed for user `e1cd100457b54531a6e9fe53001198c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:58.452684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-375e8ac2-7c87-4693-8631-231f6c169f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:58.530032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1065/2137] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:58.543061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:58.544930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:58.612062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104cf1-9467-4a5d-a35c-1662f0618cbc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:58.616300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c304-d2ee-4708-abc1-86fa3ed1c47e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:58.616935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c304-d2ee-4708-abc1-86fa3ed1c47e service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.617205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c304-d2ee-4708-abc1-86fa3ed1c47e service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.617478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c304-d2ee-4708-abc1-86fa3ed1c47e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.627554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47c304-d2ee-4708-abc1-86fa3ed1c47e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:58.629021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47c304-d2ee-4708-abc1-86fa3ed1c47e service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:58.639923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1073/2138] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:58 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:58.840278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e8ac2-7c87-4693-8631-231f6c169f0e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:58.909240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e8ac2-7c87-4693-8631-231f6c169f0e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:58.919579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:58.920340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:58.920696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:58.921063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] PATH_INFO: `/v3/users/bfc169569e1641c980526abeb3a6f463` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:58.922571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc169569e1641c980526abeb3a6f463)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:58.924672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:58.942387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:58.994006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc169569e1641c980526abeb3a6f463'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:58.994682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc169569e1641c980526abeb3a6f463'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:58.995479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc169569e1641c980526abeb3a6f463'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.026545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc169569e1641c980526abeb3a6f463'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.038002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc169569e1641c980526abeb3a6f463'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.071418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1066/2139] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:58 2022] DELETE /identity/v3/users/bfc169569e1641c980526abeb3a6f463 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:59.075994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:59.076492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:59.076785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:59.106963 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:31:59.108764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:59.150749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:59.185401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1074/2140] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:31:59.196202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:59.197830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:59.262182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a59ab-66e1-4ebb-bf4d-11b774c647b9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:59.266265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:59.267076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:59.267463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:59.267839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:59.279022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:59.280654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:59.292750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1067/2141] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:31:59.602534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:59.604238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:31:59.619322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42bafc-5bb5-4ec3-8eee-95872fe02145 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:59.624821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:59.625836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:59.626217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:59.626587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] PATH_INFO: `/v3/projects/df624255d8f3455daa0de4f9bf4d8dd9` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:59.627970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df624255d8f3455daa0de4f9bf4d8dd9)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:59.630060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:59.690378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df624255d8f3455daa0de4f9bf4d8dd9', 'request_id': 'req-dc23dc75-92cb-412f-a28f-45b84d7286d6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.697554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df624255d8f3455daa0de4f9bf4d8dd9', 'request_id': 'req-dc23dc75-92cb-412f-a28f-45b84d7286d6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.705959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df624255d8f3455daa0de4f9bf4d8dd9', 'request_id': 'req-dc23dc75-92cb-412f-a28f-45b84d7286d6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.716413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] The token cache is being invalidate because project df624255d8f3455daa0de4f9bf4d8dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:31:59.716717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.717463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.719469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1075/2142] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:31:59 2022] DELETE /identity/v3/projects/df624255d8f3455daa0de4f9bf4d8dd9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:31:59.723207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:31:59.734511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfbec7-4dea-4ba2-9037-457b8d7a5f64 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:31:59.794112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:31:59.795041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:31:59.795324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:31:59.795599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] PATH_INFO: `/v3/users/e1cd100457b54531a6e9fe53001198c4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:31:59.796777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1cd100457b54531a6e9fe53001198c4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:31:59.798762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:31:59.839878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:31:59.862605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1cd100457b54531a6e9fe53001198c4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.863733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1cd100457b54531a6e9fe53001198c4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.864435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1cd100457b54531a6e9fe53001198c4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.883240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1cd100457b54531a6e9fe53001198c4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.890609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-90d14287-0fd2-45c7-bc17-8286b2007b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1cd100457b54531a6e9fe53001198c4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:31:59.914813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1068/2143] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:31:59 2022] DELETE /identity/v3/users/e1cd100457b54531a6e9fe53001198c4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:32:00.277386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:00.279140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:00.293127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23dc75-92cb-412f-a28f-45b84d7286d6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:00.395768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:00.397055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.397367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.397648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] PATH_INFO: `/v3/projects/2546546493de4b078f61e29ffd4471d6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.398803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2546546493de4b078f61e29ffd4471d6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:00.400846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:00.455345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2546546493de4b078f61e29ffd4471d6', 'request_id': 'req-5d981c9f-053c-4edc-a439-eb0a59ac9d24'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:00.463973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2546546493de4b078f61e29ffd4471d6', 'request_id': 'req-5d981c9f-053c-4edc-a439-eb0a59ac9d24'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:00.474212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2546546493de4b078f61e29ffd4471d6', 'request_id': 'req-5d981c9f-053c-4edc-a439-eb0a59ac9d24'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:00.481110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:00.490500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] The token cache is being invalidate because project 2546546493de4b078f61e29ffd4471d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:32:00.490783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:00.491505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5d981c9f-053c-4edc-a439-eb0a59ac9d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:00.494917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1076/2144] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:32:00 2022] DELETE /identity/v3/projects/2546546493de4b078f61e29ffd4471d6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:00.499260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42acb5-76e8-4409-a7b4-ba2620903c58 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.499260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42acb5-76e8-4409-a7b4-ba2620903c58 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.499509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42acb5-76e8-4409-a7b4-ba2620903c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.521694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4d42acb5-76e8-4409-a7b4-ba2620903c58 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:00.526551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4d42acb5-76e8-4409-a7b4-ba2620903c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:00.654074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1069/2145] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:00.663060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305c0c6-537a-4650-aa70-f1d91bbb4c2f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.663347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305c0c6-537a-4650-aa70-f1d91bbb4c2f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.663623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305c0c6-537a-4650-aa70-f1d91bbb4c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.684228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f305c0c6-537a-4650-aa70-f1d91bbb4c2f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:00.685447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f305c0c6-537a-4650-aa70-f1d91bbb4c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:00.694594 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1077/2146] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:00.698960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42acb5-76e8-4409-a7b4-ba2620903c58 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:00.709566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42acb5-76e8-4409-a7b4-ba2620903c58 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:00.714164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:00.714783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.715102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.715379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.716081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:32:00.716389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:00.717994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:00.730905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1070/2147] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:32:00.735305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305c0c6-537a-4650-aa70-f1d91bbb4c2f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:00.745670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305c0c6-537a-4650-aa70-f1d91bbb4c2f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:00.756507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:00.757295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.757640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.757983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.758728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:00.760767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:00.789350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1078/2148] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:32:00 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:32:00.793659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:00.804282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dffef-2cf5-4584-902d-bc10fd22f961 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:00.861715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:00.862343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.862629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.862939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.863786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:00.866320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:00.890467 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1071/2149] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:32:00 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:32:00.894731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:00.905105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0732af-eff3-4964-95bc-4cd2a3297e20 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:00.909626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:00.910228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.910503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.910770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.911497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:00.913076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:00.923480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1079/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:00.927972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:00.938693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94ecbf-081a-4a16-9683-62a08ef7afb7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:00.944439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54319768-4575-4660-b2a6-ecdea69c348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:00.945077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54319768-4575-4660-b2a6-ecdea69c348f admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.945364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54319768-4575-4660-b2a6-ecdea69c348f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.945645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54319768-4575-4660-b2a6-ecdea69c348f admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:00.946466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54319768-4575-4660-b2a6-ecdea69c348f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:00.947966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54319768-4575-4660-b2a6-ecdea69c348f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:00.960817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1072/2151] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:00.965023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:00.975795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012df1c-aaa0-4228-9ed3-fc135922198a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:00.980894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:00.981581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:00.981844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:00.982111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] PATH_INFO: `/v3/projects/5fe38f0c19ab400cafd6ce395c52a550/users/f68d23e1c47e47acbd02ab424322d217/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.000926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe38f0c19ab400cafd6ce395c52a550, user_id=f68d23e1c47e47acbd02ab424322d217, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.003627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.022523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1080/2152] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:00 2022] PUT /identity/v3/projects/5fe38f0c19ab400cafd6ce395c52a550/users/f68d23e1c47e47acbd02ab424322d217/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:01.030509 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe036e7-a943-44d4-bd23-28900da3f84b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.030795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe036e7-a943-44d4-bd23-28900da3f84b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.031102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe036e7-a943-44d4-bd23-28900da3f84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.062147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ebe036e7-a943-44d4-bd23-28900da3f84b None None] MFA Rules not processed for user `f68d23e1c47e47acbd02ab424322d217`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:01.063479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ebe036e7-a943-44d4-bd23-28900da3f84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:01.070196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:01.142085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1073/2153] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:01.146651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.235227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644027-1057-4739-b8e9-d04ddf48700a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.240787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.241828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.242192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.242556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.243340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.245336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.282919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1081/2154] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:32:01 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:32:01.285798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe036e7-a943-44d4-bd23-28900da3f84b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.297458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe036e7-a943-44d4-bd23-28900da3f84b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.354860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.355438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.355723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.356003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.356674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.358935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.381726 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1074/2155] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:32:01 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:32:01.385853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.397286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690ce1-6e2b-425a-8c64-b3aeb89b9158 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.402191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.402785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.403104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.403378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.404076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.405689 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.419929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1082/2156] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:01.424152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.435584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab89a26-73b6-47b5-b858-8725f6f58f5a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.440830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.441506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.441787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.442062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] PATH_INFO: `/v3/projects/a360312ac43647c89b844480e0a25e70/users/9755c54382684cb0834a953484acc736/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.459094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a360312ac43647c89b844480e0a25e70, user_id=9755c54382684cb0834a953484acc736, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.461746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.476678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1075/2157] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:01 2022] PUT /identity/v3/projects/a360312ac43647c89b844480e0a25e70/users/9755c54382684cb0834a953484acc736/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:01.480653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.492124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32619c06-48bf-4b34-858d-2186cd32e81e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.571058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201a3b0-39d5-466a-9b48-610c679ae53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.571683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a3b0-39d5-466a-9b48-610c679ae53e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.571961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a3b0-39d5-466a-9b48-610c679ae53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.572234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a3b0-39d5-466a-9b48-610c679ae53e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.572916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201a3b0-39d5-466a-9b48-610c679ae53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.574491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201a3b0-39d5-466a-9b48-610c679ae53e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.581031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:01.590720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1083/2158] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:01.591816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.603122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789de1f-f8de-410a-be79-4f5734fd8b00 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.607952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.608573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.609077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.609360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] PATH_INFO: `/v3/projects/a360312ac43647c89b844480e0a25e70/users/9755c54382684cb0834a953484acc736/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.611838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a360312ac43647c89b844480e0a25e70, user_id=9755c54382684cb0834a953484acc736, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.614456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.625594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1076/2159] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:01 2022] PUT /identity/v3/projects/a360312ac43647c89b844480e0a25e70/users/9755c54382684cb0834a953484acc736/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:01.637095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.637384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.637662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.670177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] MFA Rules not processed for user `9755c54382684cb0834a953484acc736`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:01.671418 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:01.760871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1084/2160] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:01.772892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.774754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:01.845381 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095a56c-31c0-43c8-acbc-baf29fb7815b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.850023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.850727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.851610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.851610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.861269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:01.862646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:01.872976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1077/2161] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:01.920882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:01.922351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:01.934914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69be7-343e-4ee6-a938-8e6d9991c368 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:01.938722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2720e8-ec9a-4ad4-91da-844ba95caacb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:01.939432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2720e8-ec9a-4ad4-91da-844ba95caacb service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:01.939716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2720e8-ec9a-4ad4-91da-844ba95caacb service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:01.939992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2720e8-ec9a-4ad4-91da-844ba95caacb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:01.998818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2720e8-ec9a-4ad4-91da-844ba95caacb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:02.000289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2720e8-ec9a-4ad4-91da-844ba95caacb service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:02.010537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1085/2162] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:06.578357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:06.582888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:06.640833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c47265-b1df-4ddd-96cf-c74de2e00da7 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:06.645474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:06.646055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:06.646333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:06.646607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] PATH_INFO: `/v3/users/f68d23e1c47e47acbd02ab424322d217` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:06.647787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f68d23e1c47e47acbd02ab424322d217)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:06.649679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:06.693448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f68d23e1c47e47acbd02ab424322d217'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:06.694086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f68d23e1c47e47acbd02ab424322d217'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:06.694695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f68d23e1c47e47acbd02ab424322d217'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:06.706420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f68d23e1c47e47acbd02ab424322d217'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:06.713153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f68d23e1c47e47acbd02ab424322d217'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:06.725374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1078/2163] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:32:06 2022] DELETE /identity/v3/users/f68d23e1c47e47acbd02ab424322d217 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:06.730032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:06.730548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:06.730648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:06.749576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:06.750747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:06.808877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1086/2164] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:06.818460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:06.819856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:06.877253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880eba8-c2cf-4ca5-bce4-79380a112e5c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:06.880917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:06.881558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:06.881839 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:06.882113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:06.890980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:06.892398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:06.902115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1079/2165] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:07.164391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:07.165906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:07.177980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f7b35-eee8-438b-b7a6-d5d780fcc12b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:07.183417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:07.184061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:07.184345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:07.184618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] PATH_INFO: `/v3/projects/5fe38f0c19ab400cafd6ce395c52a550` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:07.185987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe38f0c19ab400cafd6ce395c52a550)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:07.187950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:07.210612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:07.234784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe38f0c19ab400cafd6ce395c52a550', 'request_id': 'req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.240680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe38f0c19ab400cafd6ce395c52a550', 'request_id': 'req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.248242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe38f0c19ab400cafd6ce395c52a550', 'request_id': 'req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.258792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] The token cache is being invalidate because project 5fe38f0c19ab400cafd6ce395c52a550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:32:07.259138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.259765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.261259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1087/2166] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:32:07 2022] DELETE /identity/v3/projects/5fe38f0c19ab400cafd6ce395c52a550 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:07.265265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:07.275378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036a407-b97f-4fc5-9338-cf58555eb90b service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:07.294885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:07.336988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:07.338091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:07.338603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:07.339134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] PATH_INFO: `/v3/users/9755c54382684cb0834a953484acc736` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:07.340343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9755c54382684cb0834a953484acc736)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:07.342271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:07.394727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9755c54382684cb0834a953484acc736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.395667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9755c54382684cb0834a953484acc736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.397073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9755c54382684cb0834a953484acc736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.412645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9755c54382684cb0834a953484acc736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.421898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-072c67fb-74fc-4043-b6cd-d1cda528145c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9755c54382684cb0834a953484acc736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.436922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1080/2167] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:32:07 2022] DELETE /identity/v3/users/9755c54382684cb0834a953484acc736 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:07.813358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:07.815202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:07.829766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f128-1b7f-40be-8fc4-d6d1be588b4a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:07.897856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:07.898493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:07.898771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:07.899086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] PATH_INFO: `/v3/projects/a360312ac43647c89b844480e0a25e70` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:07.900232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a360312ac43647c89b844480e0a25e70)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:07.902111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:07.947191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a360312ac43647c89b844480e0a25e70', 'request_id': 'req-5b14f630-d874-48a4-9671-03ba0698c440'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.953006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a360312ac43647c89b844480e0a25e70', 'request_id': 'req-5b14f630-d874-48a4-9671-03ba0698c440'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.960201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a360312ac43647c89b844480e0a25e70', 'request_id': 'req-5b14f630-d874-48a4-9671-03ba0698c440'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.971094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] The token cache is being invalidate because project a360312ac43647c89b844480e0a25e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:32:07.971383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.972011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b14f630-d874-48a4-9671-03ba0698c440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:07.973483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1088/2168] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:32:07 2022] DELETE /identity/v3/projects/a360312ac43647c89b844480e0a25e70 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:07.979680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda2eac-706b-4cbc-aa80-51ac1b9d91d7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:07.980138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda2eac-706b-4cbc-aa80-51ac1b9d91d7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:07.980237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda2eac-706b-4cbc-aa80-51ac1b9d91d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:07.999711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0eda2eac-706b-4cbc-aa80-51ac1b9d91d7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:08.000904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0eda2eac-706b-4cbc-aa80-51ac1b9d91d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:08.064910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1081/2169] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:08.072661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398cfc-6f76-4b6d-8e41-f5f136c3e5a9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.072943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398cfc-6f76-4b6d-8e41-f5f136c3e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.073216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398cfc-6f76-4b6d-8e41-f5f136c3e5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.092363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d4398cfc-6f76-4b6d-8e41-f5f136c3e5a9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:08.093637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d4398cfc-6f76-4b6d-8e41-f5f136c3e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:08.102560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1089/2170] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:08.106708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda2eac-706b-4cbc-aa80-51ac1b9d91d7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.116856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda2eac-706b-4cbc-aa80-51ac1b9d91d7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.121391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.121993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.122272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.122547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.123273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:32:08.123577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.125147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.136532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1082/2171] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:32:08.140550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4398cfc-6f76-4b6d-8e41-f5f136c3e5a9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.149573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4398cfc-6f76-4b6d-8e41-f5f136c3e5a9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.154208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.154802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.155114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.155384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.156040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.157918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.186310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1090/2172] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:32:08.190456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.200746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0542d2-31a2-4300-b25c-40de71e63a46 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.231263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:08.262791 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.263445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.263727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.264005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.264666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.266574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.289366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1083/2173] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:32:08.293535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.306616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64011f-5a63-4794-b16a-2481a838e648 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.311637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.312270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.312548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.312818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.313514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.315129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.317575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:08.330286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1091/2174] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:08.331614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.342146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186c21-f4a3-4f37-8011-1cca7d9d243c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.346776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d14f0-de60-4a0d-afcb-b6cb868ab83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.347422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d14f0-de60-4a0d-afcb-b6cb868ab83f admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.347705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d14f0-de60-4a0d-afcb-b6cb868ab83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.347982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d14f0-de60-4a0d-afcb-b6cb868ab83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.348702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d14f0-de60-4a0d-afcb-b6cb868ab83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.350962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d14f0-de60-4a0d-afcb-b6cb868ab83f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.361573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1084/2175] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:32:08.365645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.375526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532146c-df57-442c-adcd-304996fcdaf4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.380396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.381496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.382027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.382451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] PATH_INFO: `/v3/projects/bc4f4dca53d44cd9b2df3f7132181cd8/users/be41876344a04d369fe6d6a5ce791717/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.399110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4f4dca53d44cd9b2df3f7132181cd8, user_id=be41876344a04d369fe6d6a5ce791717, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.401759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.416156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1092/2176] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:08 2022] PUT /identity/v3/projects/bc4f4dca53d44cd9b2df3f7132181cd8/users/be41876344a04d369fe6d6a5ce791717/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:08.426374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebef4c-c978-4bf0-b9ce-839ac15934a6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.426666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebef4c-c978-4bf0-b9ce-839ac15934a6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.426986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebef4c-c978-4bf0-b9ce-839ac15934a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.453740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-04ebef4c-c978-4bf0-b9ce-839ac15934a6 None None] MFA Rules not processed for user `be41876344a04d369fe6d6a5ce791717`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:08.454917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04ebef4c-c978-4bf0-b9ce-839ac15934a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:08.518491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1085/2177] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:08.523581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.586306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4d0d4-8fdd-4826-a25c-cd7b07c7a9bc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.591028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.591647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.591950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.592249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.592914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.594902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.623789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1093/2178] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:32:08.627738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebef4c-c978-4bf0-b9ce-839ac15934a6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.639896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebef4c-c978-4bf0-b9ce-839ac15934a6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.703444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.704135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.704355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.704629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.705285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.707215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.728201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1086/2179] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:32:08.732158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.742322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd63bf-60fe-47cc-bac3-4eb92b4c43e2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.746887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.747493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.747771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.748042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.748724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.750306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.761064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1094/2180] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:08.764892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.775314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7d00-a5fa-4e92-9686-25011a977f0b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.779934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa7f98-ec9d-4c83-b282-67452230b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.780529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa7f98-ec9d-4c83-b282-67452230b961 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.780809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa7f98-ec9d-4c83-b282-67452230b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.781082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa7f98-ec9d-4c83-b282-67452230b961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.781766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa7f98-ec9d-4c83-b282-67452230b961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.783532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa7f98-ec9d-4c83-b282-67452230b961 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.793376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1087/2181] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:32:08.797538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:08.807780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956065-8994-4803-a2ab-6e05ff756ed3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:08.812493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:08.813146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.813458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.814012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] PATH_INFO: `/v3/projects/cd1c92b06c3a4ea1ba601a15c38f70b7/users/4892c796383540c4b09a7163ef9b7eba/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.831697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1c92b06c3a4ea1ba601a15c38f70b7, user_id=4892c796383540c4b09a7163ef9b7eba, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:08.835118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:08.850473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1095/2182] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:08 2022] PUT /identity/v3/projects/cd1c92b06c3a4ea1ba601a15c38f70b7/users/4892c796383540c4b09a7163ef9b7eba/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:32:08.861542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eabc50-b9d3-4fe1-8b09-db9a679e4500 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:08.861835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eabc50-b9d3-4fe1-8b09-db9a679e4500 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:08.862115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eabc50-b9d3-4fe1-8b09-db9a679e4500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:08.891508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-84eabc50-b9d3-4fe1-8b09-db9a679e4500 None None] MFA Rules not processed for user `4892c796383540c4b09a7163ef9b7eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:08.893041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-84eabc50-b9d3-4fe1-8b09-db9a679e4500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:08.899496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:08.963352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1088/2183] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:08.968018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:09.032389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d60988-ac96-4b55-a802-2203e16e8358 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:09.037276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:09.037955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.039749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.040476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.041848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:09.044250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:09.080912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1096/2184] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:32:08 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:32:09.082318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eabc50-b9d3-4fe1-8b09-db9a679e4500 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:09.095404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eabc50-b9d3-4fe1-8b09-db9a679e4500 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:09.176355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:09.177607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.177890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.178309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.179241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:09.181566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:09.212749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1089/2185] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:32:09 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:32:09.217094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:09.229895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec791c-cc25-401e-8157-d17c70c29ceb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:09.234780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:09.235506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.235844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.236186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.236988 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:09.238687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:09.244406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:09.259169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1097/2186] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:09.260394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:09.271404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aed9a8-746c-4b4d-abda-bcd9191e83b5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:09.277874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:09.278692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.279151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.279537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] PATH_INFO: `/v3/projects/1b582a2f6a404d628a25cac4dfe2ec3a/users/db6c7c7301e9427e98b6d3f9bc2f2991/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.298363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b582a2f6a404d628a25cac4dfe2ec3a, user_id=db6c7c7301e9427e98b6d3f9bc2f2991, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:09.301308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:09.319781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1090/2187] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:09 2022] PUT /identity/v3/projects/1b582a2f6a404d628a25cac4dfe2ec3a/users/db6c7c7301e9427e98b6d3f9bc2f2991/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:09.323793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:09.335879 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0667bce-6fac-40c1-b33b-610962d052f1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:09.397585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973641f-e8d4-4245-8785-44036a723b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:09.398206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973641f-e8d4-4245-8785-44036a723b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.398485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973641f-e8d4-4245-8785-44036a723b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.398789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973641f-e8d4-4245-8785-44036a723b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.399582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973641f-e8d4-4245-8785-44036a723b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:09.401172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973641f-e8d4-4245-8785-44036a723b5d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:09.410929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1098/2188] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:09.415670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:09.427765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e9aa0-4b0a-47d5-915c-52f3d969829d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:09.432747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:09.433413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.433692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.433963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] PATH_INFO: `/v3/projects/1b582a2f6a404d628a25cac4dfe2ec3a/users/db6c7c7301e9427e98b6d3f9bc2f2991/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.435962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b582a2f6a404d628a25cac4dfe2ec3a, user_id=db6c7c7301e9427e98b6d3f9bc2f2991, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:09.438571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:09.449363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1091/2189] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:09 2022] PUT /identity/v3/projects/1b582a2f6a404d628a25cac4dfe2ec3a/users/db6c7c7301e9427e98b6d3f9bc2f2991/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:09.459623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.459914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.460191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.488924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] MFA Rules not processed for user `db6c7c7301e9427e98b6d3f9bc2f2991`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:09.490695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:09.561528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1099/2190] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:09.572821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:09.574548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:09.634547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273bc0a-c234-4f68-a556-ea1979cb979b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:09.638789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:09.639440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:09.639719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:09.639988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:09.655801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:09.712742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:09.714634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:09.725580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1092/2191] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:11.268116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:11.269526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:11.281814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b623aeb-a817-41bf-9566-cb844af8dbae None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:11.285389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:11.286002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:11.286959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:11.287240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:11.352740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:11.354194 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:11.363267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1100/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:11.922992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:11.933408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1fae5-eecc-4e3d-a332-86af65ff9f6d service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:11.937587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de538e3-f8c3-44b2-9787-10697ecbefad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:11.938212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de538e3-f8c3-44b2-9787-10697ecbefad service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:11.938492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de538e3-f8c3-44b2-9787-10697ecbefad service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:11.938756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de538e3-f8c3-44b2-9787-10697ecbefad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:11.950313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de538e3-f8c3-44b2-9787-10697ecbefad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:11.951188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de538e3-f8c3-44b2-9787-10697ecbefad service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:11.962220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1093/2193] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:15.981485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:16.001169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:16.047624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925e767-869b-4033-801f-d13e46d45c3c service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:16.052089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:16.052743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:16.053025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:16.053301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] PATH_INFO: `/v3/users/be41876344a04d369fe6d6a5ce791717` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:16.054466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] RBAC: Authorizing `identity:delete_user(user_id=be41876344a04d369fe6d6a5ce791717)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:16.056703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:16.103105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be41876344a04d369fe6d6a5ce791717'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.109820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be41876344a04d369fe6d6a5ce791717'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.110533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be41876344a04d369fe6d6a5ce791717'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.111175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be41876344a04d369fe6d6a5ce791717'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.119829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be41876344a04d369fe6d6a5ce791717'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.139844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1101/2194] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:32:15 2022] DELETE /identity/v3/users/be41876344a04d369fe6d6a5ce791717 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:16.144219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:16.144727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:16.144837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:16.164624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:16.165824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:16.229518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1094/2195] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:32:16.239733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:16.241088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:16.299377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6eab01-927f-4ccb-9342-e288baaf879e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:16.303199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:16.303815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:16.304090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:16.304443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:16.314511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:16.316018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:16.326907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1102/2196] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:16.631541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:16.631541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:16.639059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664557-a414-4f0e-99f4-752ed3fbdcf8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:16.644244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:16.644908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:16.645356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:16.645656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] PATH_INFO: `/v3/projects/bc4f4dca53d44cd9b2df3f7132181cd8` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:16.647042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4f4dca53d44cd9b2df3f7132181cd8)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:16.649085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:16.696626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4f4dca53d44cd9b2df3f7132181cd8', 'request_id': 'req-20868f8a-48e6-4bdc-b71f-8fec99608fa8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.702893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4f4dca53d44cd9b2df3f7132181cd8', 'request_id': 'req-20868f8a-48e6-4bdc-b71f-8fec99608fa8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.709207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4f4dca53d44cd9b2df3f7132181cd8', 'request_id': 'req-20868f8a-48e6-4bdc-b71f-8fec99608fa8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.721459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] The token cache is being invalidate because project bc4f4dca53d44cd9b2df3f7132181cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:32:16.721743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.722357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.724574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1095/2197] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:32:16 2022] DELETE /identity/v3/projects/bc4f4dca53d44cd9b2df3f7132181cd8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:16.728653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:16.738931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86571250-bc55-4c6f-b312-7792c5ab23e9 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:16.776584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:16.797003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:16.797641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:16.797918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:16.798189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] PATH_INFO: `/v3/users/4892c796383540c4b09a7163ef9b7eba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:16.799367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4892c796383540c4b09a7163ef9b7eba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:16.801278 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:16.854681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4892c796383540c4b09a7163ef9b7eba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.861647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4892c796383540c4b09a7163ef9b7eba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.862364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4892c796383540c4b09a7163ef9b7eba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.862999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4892c796383540c4b09a7163ef9b7eba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.875927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4892c796383540c4b09a7163ef9b7eba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:16.895098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1103/2198] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:32:16 2022] DELETE /identity/v3/users/4892c796383540c4b09a7163ef9b7eba => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:17.191046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:17.193000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:17.206492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868f8a-48e6-4bdc-b71f-8fec99608fa8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:17.231058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:17.269698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:17.270428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:17.271329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:17.271605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] PATH_INFO: `/v3/projects/cd1c92b06c3a4ea1ba601a15c38f70b7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:17.272831 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1c92b06c3a4ea1ba601a15c38f70b7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:17.274733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:17.320378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1c92b06c3a4ea1ba601a15c38f70b7', 'request_id': 'req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.327046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1c92b06c3a4ea1ba601a15c38f70b7', 'request_id': 'req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.333822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1c92b06c3a4ea1ba601a15c38f70b7', 'request_id': 'req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.346744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] The token cache is being invalidate because project cd1c92b06c3a4ea1ba601a15c38f70b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:32:17.347062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.347708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.349327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1096/2199] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:32:17 2022] DELETE /identity/v3/projects/cd1c92b06c3a4ea1ba601a15c38f70b7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:17.353197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:17.354631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:17.368144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3784a8-aba6-4797-a6ed-845f3b0f4c88 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:17.428014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:17.428658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:17.428940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:17.429219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] PATH_INFO: `/v3/users/db6c7c7301e9427e98b6d3f9bc2f2991` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:17.430371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6c7c7301e9427e98b6d3f9bc2f2991)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:17.432687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:17.481269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6c7c7301e9427e98b6d3f9bc2f2991'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.488347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6c7c7301e9427e98b6d3f9bc2f2991'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.489711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6c7c7301e9427e98b6d3f9bc2f2991'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.490384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6c7c7301e9427e98b6d3f9bc2f2991'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.494011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:17.498453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-173f83db-557f-4ba3-8879-7f1e8694a0da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6c7c7301e9427e98b6d3f9bc2f2991'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:17.518533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1104/2200] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:32:17 2022] DELETE /identity/v3/users/db6c7c7301e9427e98b6d3f9bc2f2991 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:17.878981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:17.880472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:17.898202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae097-dd39-4113-b1dd-f25f32a78ddb admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:17.960418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:17.961064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:17.961344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:17.961618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] PATH_INFO: `/v3/projects/1b582a2f6a404d628a25cac4dfe2ec3a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:17.962800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b582a2f6a404d628a25cac4dfe2ec3a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:17.964774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.012825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b582a2f6a404d628a25cac4dfe2ec3a', 'request_id': 'req-5d0977c4-8db8-4afd-af39-009207e293e3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:18.018753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b582a2f6a404d628a25cac4dfe2ec3a', 'request_id': 'req-5d0977c4-8db8-4afd-af39-009207e293e3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:18.021205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:18.027478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b582a2f6a404d628a25cac4dfe2ec3a', 'request_id': 'req-5d0977c4-8db8-4afd-af39-009207e293e3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:18.039982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] The token cache is being invalidate because project 1b582a2f6a404d628a25cac4dfe2ec3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:32:18.040271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:18.040913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5d0977c4-8db8-4afd-af39-009207e293e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:32:18.042551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1097/2201] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:32:17 2022] DELETE /identity/v3/projects/1b582a2f6a404d628a25cac4dfe2ec3a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:18.047567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd78f16-af21-4e65-a254-ad758f68091d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.047855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd78f16-af21-4e65-a254-ad758f68091d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.048129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd78f16-af21-4e65-a254-ad758f68091d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.067739 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7dd78f16-af21-4e65-a254-ad758f68091d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:18.069238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7dd78f16-af21-4e65-a254-ad758f68091d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:18.129392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1105/2202] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:18.137698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23535eba-1a4d-47f1-b769-33ebd4e1d376 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.137985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23535eba-1a4d-47f1-b769-33ebd4e1d376 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.138263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23535eba-1a4d-47f1-b769-33ebd4e1d376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.159695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-23535eba-1a4d-47f1-b769-33ebd4e1d376 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:18.161648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-23535eba-1a4d-47f1-b769-33ebd4e1d376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:18.169585 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1098/2203] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:18.173763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd78f16-af21-4e65-a254-ad758f68091d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:18.184482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd78f16-af21-4e65-a254-ad758f68091d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:18.189548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:18.190154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.190432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.190703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.191431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:32:18.191735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:18.193329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.209277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1106/2204] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:32:18.213648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23535eba-1a4d-47f1-b769-33ebd4e1d376 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:18.223082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23535eba-1a4d-47f1-b769-33ebd4e1d376 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:18.228336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609b956-db74-451d-8822-143252f2032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:18.229078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609b956-db74-451d-8822-143252f2032d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.229363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609b956-db74-451d-8822-143252f2032d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.229641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609b956-db74-451d-8822-143252f2032d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.230323 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609b956-db74-451d-8822-143252f2032d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:18.232291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609b956-db74-451d-8822-143252f2032d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.263174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1099/2205] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:32:18 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:32:18.267432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:18.279676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297c290-e091-491d-b9ad-0fbe0689e823 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:18.340706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:18.341327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.341605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.341883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.342544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:18.344490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.365917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1107/2206] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:32:18 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:32:18.370024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609b956-db74-451d-8822-143252f2032d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:18.380860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609b956-db74-451d-8822-143252f2032d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:18.385414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:18.386014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.386302 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.386571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.387297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:18.388863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.400827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1100/2207] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:18.405103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:18.415185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a86cc-71f5-45bd-835f-2af350f4f249 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:18.420453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee831d-a4c7-4d48-8814-50ef88c23b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:18.421058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee831d-a4c7-4d48-8814-50ef88c23b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.421337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee831d-a4c7-4d48-8814-50ef88c23b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.421609 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee831d-a4c7-4d48-8814-50ef88c23b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.422292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee831d-a4c7-4d48-8814-50ef88c23b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:18.423979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee831d-a4c7-4d48-8814-50ef88c23b3d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.437094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1108/2208] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:32:18.441333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:18.456709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46969a65-a195-48c8-82f2-151d7247a512 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:18.462471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:18.463181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.463459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.463728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] PATH_INFO: `/v3/projects/917feaf8947e4203ae71b81448fb898c/users/c7368e4beec0418e9ae2103f34d1f504/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.478280 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=917feaf8947e4203ae71b81448fb898c, user_id=c7368e4beec0418e9ae2103f34d1f504, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:18.480913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.493469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1101/2209] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:32:18 2022] PUT /identity/v3/projects/917feaf8947e4203ae71b81448fb898c/users/c7368e4beec0418e9ae2103f34d1f504/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:32:18.505983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a899d-a2d6-4e1e-86e8-e10f2017504c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.506347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a899d-a2d6-4e1e-86e8-e10f2017504c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.506724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a899d-a2d6-4e1e-86e8-e10f2017504c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.536144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8b6a899d-a2d6-4e1e-86e8-e10f2017504c None None] MFA Rules not processed for user `c7368e4beec0418e9ae2103f34d1f504`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:18.537345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8b6a899d-a2d6-4e1e-86e8-e10f2017504c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:18.571780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:18.607485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1109/2210] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:18.619187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:18.621011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:18.678587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb769085-6909-4451-b3f5-4cf354e208bf admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:18.682480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:18.683214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:18.683536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:18.683829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:18.693074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:18.694480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:18.703278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1102/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:28.604938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=12 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:28.704091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=57,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:28.975137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fc170-2484-470a-9381-97463d17849b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:28.975427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fc170-2484-470a-9381-97463d17849b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:28.975702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fc170-2484-470a-9381-97463d17849b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:28.995220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-600fc170-2484-470a-9381-97463d17849b None None] MFA Rules not processed for user `4dadf737392d4153ae50de020534f070`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:32:28.996413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-600fc170-2484-470a-9381-97463d17849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:29.048607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1110/2212] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:32:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:32:39.042076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=25 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:32:41.260098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:32:41.261571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:41.324397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960419e-80a8-46d0-99cc-7d0cee428b03 service glance] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:32:41.327992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:32:41.328598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:32:41.328876 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:32:41.329149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:32:41.331061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:32:41.390680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:32:41.392110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:32:41.401428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1103/2213] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:32:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:32:51.397169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=50 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:33:03.416719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fc170-2484-470a-9381-97463d17849b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:03.418141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-600fc170-2484-470a-9381-97463d17849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:03.487515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fc170-2484-470a-9381-97463d17849b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:03.492342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba62d4-25d9-47bd-96dc-8d06bb03d3b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:03.492951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba62d4-25d9-47bd-96dc-8d06bb03d3b8 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:03.493228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba62d4-25d9-47bd-96dc-8d06bb03d3b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:03.493501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba62d4-25d9-47bd-96dc-8d06bb03d3b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:03.495259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a1ba62d4-25d9-47bd-96dc-8d06bb03d3b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:03.506080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba62d4-25d9-47bd-96dc-8d06bb03d3b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:03.508160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba62d4-25d9-47bd-96dc-8d06bb03d3b8 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:03.517588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1111/2214] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:33:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:33:13.513667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=32 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:33:40.511860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:40.573199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a06ee-2af7-4326-bfee-213917f895af service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:40.577713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:40.578455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:40.578737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:40.579041 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] PATH_INFO: `/v3/users/c7368e4beec0418e9ae2103f34d1f504` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:40.580200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7368e4beec0418e9ae2103f34d1f504)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:40.582086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:40.627298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7368e4beec0418e9ae2103f34d1f504'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:40.627971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7368e4beec0418e9ae2103f34d1f504'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:40.628599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7368e4beec0418e9ae2103f34d1f504'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:40.640292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7368e4beec0418e9ae2103f34d1f504'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:40.646995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7368e4beec0418e9ae2103f34d1f504'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:40.660026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1104/2215] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:33:40 2022] DELETE /identity/v3/users/c7368e4beec0418e9ae2103f34d1f504 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:33:40.664636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:40.665149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:40.665248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:40.684624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:33:40.685809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:40.745615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1112/2216] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:33:40.755350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:40.756759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:40.813433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132412f4-2edb-4264-ad2c-4a03a1c91672 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:40.817023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6ecd5-679d-4f06-8f6b-705559813346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:40.817631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6ecd5-679d-4f06-8f6b-705559813346 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:40.817909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6ecd5-679d-4f06-8f6b-705559813346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:40.818182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6ecd5-679d-4f06-8f6b-705559813346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:40.827497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6ecd5-679d-4f06-8f6b-705559813346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:40.828899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6ecd5-679d-4f06-8f6b-705559813346 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:40.838712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1105/2217] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:33:41.107774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.109193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:41.121657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdd6d2-b085-443a-9c47-fc7a6f860775 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.127012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.127883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.128161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.128429 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] PATH_INFO: `/v3/projects/917feaf8947e4203ae71b81448fb898c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.129820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=917feaf8947e4203ae71b81448fb898c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.132309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.178589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917feaf8947e4203ae71b81448fb898c', 'request_id': 'req-9fbb9523-d1c8-4192-91fd-663a2e0afafd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:41.186049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917feaf8947e4203ae71b81448fb898c', 'request_id': 'req-9fbb9523-d1c8-4192-91fd-663a2e0afafd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:41.192676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917feaf8947e4203ae71b81448fb898c', 'request_id': 'req-9fbb9523-d1c8-4192-91fd-663a2e0afafd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:41.203391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] The token cache is being invalidate because project 917feaf8947e4203ae71b81448fb898c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:33:41.203680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:41.204312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fbb9523-d1c8-4192-91fd-663a2e0afafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:33:41.205824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1113/2218] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:33:41 2022] DELETE /identity/v3/projects/917feaf8947e4203ae71b81448fb898c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:33:41.210647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ac5fc-8358-4a7f-b80e-18cf59c4eba4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.210960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ac5fc-8358-4a7f-b80e-18cf59c4eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.211239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ac5fc-8358-4a7f-b80e-18cf59c4eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.230163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a93ac5fc-8358-4a7f-b80e-18cf59c4eba4 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:33:41.231379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a93ac5fc-8358-4a7f-b80e-18cf59c4eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:41.248722 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:33:41.300647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1106/2219] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:33:41.310428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d4685-1407-40c3-b3b3-c726389edab7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.310713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d4685-1407-40c3-b3b3-c726389edab7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.311026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d4685-1407-40c3-b3b3-c726389edab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.331468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ca7d4685-1407-40c3-b3b3-c726389edab7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:33:41.332752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ca7d4685-1407-40c3-b3b3-c726389edab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:41.341357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1114/2220] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:33:41.346198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ac5fc-8358-4a7f-b80e-18cf59c4eba4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.355810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ac5fc-8358-4a7f-b80e-18cf59c4eba4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.360347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.360976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.361250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.361519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.362201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:33:41.362502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.364104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.375843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1107/2221] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:33:41.379900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d4685-1407-40c3-b3b3-c726389edab7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.389606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d4685-1407-40c3-b3b3-c726389edab7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.394270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.394885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.395183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.395456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.396119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.398039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.430678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1115/2222] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:33:41 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:33:41.435643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.445515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6b2d-6817-4d60-a3d0-8da88aed9fec admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.503971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.504600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.504912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.505186 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.505856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.507822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.529269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1108/2223] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:33:41 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:33:41.533302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.545565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3483418-92bc-455a-9989-4252dbea351b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.550535 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.551220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.551626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.551906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.552600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.554192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.567308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1116/2224] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:33:41.571049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.581277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2a012-d469-402a-a3cc-c0d83ca506a8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.586599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96ce46-1087-426d-aa98-6807c86bb085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.587355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96ce46-1087-426d-aa98-6807c86bb085 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.587992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96ce46-1087-426d-aa98-6807c86bb085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.588337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96ce46-1087-426d-aa98-6807c86bb085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.589042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96ce46-1087-426d-aa98-6807c86bb085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.590658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96ce46-1087-426d-aa98-6807c86bb085 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.601460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1109/2225] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:33:41.606415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.619064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7024ea-e488-4a7d-b072-c93aa2cf8556 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.624791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.625698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.626191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.626718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] PATH_INFO: `/v3/projects/f1f35969379f41738710334f3e8438f0/users/42f551bd8c7842b4b4476c74f0f4a9e3/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.645129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f35969379f41738710334f3e8438f0, user_id=42f551bd8c7842b4b4476c74f0f4a9e3, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.649480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.665566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1117/2226] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:33:41 2022] PUT /identity/v3/projects/f1f35969379f41738710334f3e8438f0/users/42f551bd8c7842b4b4476c74f0f4a9e3/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:33:41.676868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306bf27-08b8-4efd-aac9-a170f764b6b7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.677241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306bf27-08b8-4efd-aac9-a170f764b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.677584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306bf27-08b8-4efd-aac9-a170f764b6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.707452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3306bf27-08b8-4efd-aac9-a170f764b6b7 None None] MFA Rules not processed for user `42f551bd8c7842b4b4476c74f0f4a9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:33:41.708659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3306bf27-08b8-4efd-aac9-a170f764b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:41.774246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1110/2227] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:33:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:33:41.786588 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:33:41.788311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:41.809657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:33:41.848892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9d84f-f812-4ed5-af43-bcb5aab8e53b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:33:41.852863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9e2bc-c037-4ac8-9eef-dbed2e54a777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:33:41.853531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9e2bc-c037-4ac8-9eef-dbed2e54a777 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:41.853808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9e2bc-c037-4ac8-9eef-dbed2e54a777 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:41.854078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9e2bc-c037-4ac8-9eef-dbed2e54a777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:41.862528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9e2bc-c037-4ac8-9eef-dbed2e54a777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:33:41.864017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9e2bc-c037-4ac8-9eef-dbed2e54a777 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:33:41.873245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1118/2228] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:33:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:33:44.898237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dca77-2f4a-4d47-8f42-0637b224da07 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:33:44.898690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dca77-2f4a-4d47-8f42-0637b224da07 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:33:44.898783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dca77-2f4a-4d47-8f42-0637b224da07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:33:44.917040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5d8dca77-2f4a-4d47-8f42-0637b224da07 None None] MFA Rules not processed for user `5378c85cd31c42dfbc39ce738a8b6dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:33:44.918260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5d8dca77-2f4a-4d47-8f42-0637b224da07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:33:44.929510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:33:44.978094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1111/2229] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:33:51.868683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=17 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:33:54.971399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=17 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:07.075500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:07.075984 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:07.076172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:07.096222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:07.097452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:07.176917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1119/2230] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:34:07.195159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef539194-501c-4686-9f3c-98360026c461 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:07.195847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef539194-501c-4686-9f3c-98360026c461 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:07.195847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef539194-501c-4686-9f3c-98360026c461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:07.223326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ef539194-501c-4686-9f3c-98360026c461 None None] MFA Rules not processed for user `eb6f411ed91946eba54e46799edb12d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:07.224609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ef539194-501c-4686-9f3c-98360026c461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:07.247063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:07.248490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:07.292430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1112/2231] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:34:07.337295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13d65f-c8ba-44df-81a1-9effbdfa680e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:07.348316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13e673-a2d6-4865-97e5-0a36dcbd6215 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:07.348990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e673-a2d6-4865-97e5-0a36dcbd6215 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:07.349272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e673-a2d6-4865-97e5-0a36dcbd6215 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:07.349545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e673-a2d6-4865-97e5-0a36dcbd6215 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:07.359366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13e673-a2d6-4865-97e5-0a36dcbd6215 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:07.361002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13e673-a2d6-4865-97e5-0a36dcbd6215 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:07.370498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1120/2232] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:34:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 14:34:17.285134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=25 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:17.366354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=57 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:28.237790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47c66-5b9a-41b9-8792-65e0357c0c81 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.238387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47c66-5b9a-41b9-8792-65e0357c0c81 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.238387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47c66-5b9a-41b9-8792-65e0357c0c81 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.239767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1113/2233] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:34:28.245983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af81ac-34c2-4dea-9d1f-78f79ebcdcde None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.246558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af81ac-34c2-4dea-9d1f-78f79ebcdcde None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.247395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af81ac-34c2-4dea-9d1f-78f79ebcdcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.268654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-39af81ac-34c2-4dea-9d1f-78f79ebcdcde None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:28.269993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-39af81ac-34c2-4dea-9d1f-78f79ebcdcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:28.351149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1121/2234] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:34:28.368013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf47c66-5b9a-41b9-8792-65e0357c0c81 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:28.370339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aaf47c66-5b9a-41b9-8792-65e0357c0c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:28.436290 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf47c66-5b9a-41b9-8792-65e0357c0c81 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:28.439955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5405-8abe-4231-8294-7fd1cffcb19f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:28.440578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5405-8abe-4231-8294-7fd1cffcb19f service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.440862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5405-8abe-4231-8294-7fd1cffcb19f service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.441138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5405-8abe-4231-8294-7fd1cffcb19f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.452573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5405-8abe-4231-8294-7fd1cffcb19f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:28.454361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5405-8abe-4231-8294-7fd1cffcb19f service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:28.465659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1114/2235] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:34:28.545027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7142fb-dc1f-480d-88a8-95cb1d5cf3a5 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.545387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7142fb-dc1f-480d-88a8-95cb1d5cf3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.545736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7142fb-dc1f-480d-88a8-95cb1d5cf3a5 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.547229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1122/2236] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:34:28.552234 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e54a0-c4c9-4fad-90ad-ecb8a4dea757 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.552596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e54a0-c4c9-4fad-90ad-ecb8a4dea757 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.552872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e54a0-c4c9-4fad-90ad-ecb8a4dea757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.574300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d19e54a0-c4c9-4fad-90ad-ecb8a4dea757 None None] MFA Rules not processed for user `cd235744a7ee462eb13e383930de4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:28.575529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d19e54a0-c4c9-4fad-90ad-ecb8a4dea757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:28.625268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1115/2237] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:34:28.637557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7142fb-dc1f-480d-88a8-95cb1d5cf3a5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:28.639243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9c7142fb-dc1f-480d-88a8-95cb1d5cf3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:28.652384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7142fb-dc1f-480d-88a8-95cb1d5cf3a5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:28.655948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c926d1e-5ffc-453d-9dff-5be5451b83ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:28.656553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c926d1e-5ffc-453d-9dff-5be5451b83ec service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.656859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c926d1e-5ffc-453d-9dff-5be5451b83ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.657132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c926d1e-5ffc-453d-9dff-5be5451b83ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.664090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c926d1e-5ffc-453d-9dff-5be5451b83ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:28.665999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c926d1e-5ffc-453d-9dff-5be5451b83ec service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:28.674071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1123/2238] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:34:28.762277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0fbfe-c3a6-4b9d-8a1f-86b5463122f1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.762567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0fbfe-c3a6-4b9d-8a1f-86b5463122f1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.762846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0fbfe-c3a6-4b9d-8a1f-86b5463122f1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.764244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1116/2239] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:34:28.769212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a82b3-2ad9-434a-b4c7-28c157c33fe3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.769451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a82b3-2ad9-434a-b4c7-28c157c33fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.769729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a82b3-2ad9-434a-b4c7-28c157c33fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.788926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d01a82b3-2ad9-434a-b4c7-28c157c33fe3 None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:28.790169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d01a82b3-2ad9-434a-b4c7-28c157c33fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:28.799883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1124/2240] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:34:28.811162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0fbfe-c3a6-4b9d-8a1f-86b5463122f1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:28.822660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0fbfe-c3a6-4b9d-8a1f-86b5463122f1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:28.826347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043e2a6-62a2-4bc3-8b4d-4619d7be067a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:28.826984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043e2a6-62a2-4bc3-8b4d-4619d7be067a service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:28.827267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043e2a6-62a2-4bc3-8b4d-4619d7be067a service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:28.827539 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043e2a6-62a2-4bc3-8b4d-4619d7be067a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:28.837476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043e2a6-62a2-4bc3-8b4d-4619d7be067a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:28.838913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043e2a6-62a2-4bc3-8b4d-4619d7be067a service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:28.851069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1117/2241] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:34:38.784233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=40 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:38.846032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=55 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:44.811801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a82b3-2ad9-434a-b4c7-28c157c33fe3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:44.822018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a82b3-2ad9-434a-b4c7-28c157c33fe3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:44.826517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:44.827155 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:44.827432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:44.827701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:44.829514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:44.843276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:44.844722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:44.856067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1125/2242] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:34:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:34:44.958088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:44.958581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:44.958663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:44.978034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:44.979247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:44.986948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1118/2243] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:34:44.999756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:45.011456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac95fa2-acaa-4c42-8ec4-a6c677acd528 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:45.016509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:45.017121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:45.017405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:45.017685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:45.026014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:45.027711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:45.036133 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1126/2244] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:34:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:34:46.048256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:46.049710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:46.061076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60406e-3bcb-497b-a4dd-68f15c23718f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:46.064556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d8af2-0448-4995-8187-5c093a0422b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:46.065161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d8af2-0448-4995-8187-5c093a0422b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:46.065441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d8af2-0448-4995-8187-5c093a0422b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:46.065712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d8af2-0448-4995-8187-5c093a0422b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:46.067412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ec7d8af2-0448-4995-8187-5c093a0422b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:46.080739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d8af2-0448-4995-8187-5c093a0422b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:46.082181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d8af2-0448-4995-8187-5c093a0422b5 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:46.093458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1119/2245] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:34:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:34:55.032773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=12 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:56.088429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=9 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:57.470779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:57.535247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa109-a57f-4a88-93e7-2b84ed386552 service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:57.539805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:57.540448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:57.540723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:57.540994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] PATH_INFO: `/v3/users/42f551bd8c7842b4b4476c74f0f4a9e3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:57.542290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f551bd8c7842b4b4476c74f0f4a9e3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:57.544235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:57.592936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f551bd8c7842b4b4476c74f0f4a9e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:57.599891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f551bd8c7842b4b4476c74f0f4a9e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:57.600582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f551bd8c7842b4b4476c74f0f4a9e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:57.601206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f551bd8c7842b4b4476c74f0f4a9e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:57.608451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f551bd8c7842b4b4476c74f0f4a9e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:57.626734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1127/2246] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:34:57 2022] DELETE /identity/v3/users/42f551bd8c7842b4b4476c74f0f4a9e3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:34:57.631260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:57.631772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:57.631876 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:57.650953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:57.652147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:57.711237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1120/2247] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:34:57.721701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:57.723166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:57.781346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be78620-ff07-4f28-ae33-014d0f7abe87 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:57.785244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b9aa5-4d94-4e51-9de8-afb376d15b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:57.785886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b9aa5-4d94-4e51-9de8-afb376d15b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:57.786167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b9aa5-4d94-4e51-9de8-afb376d15b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:57.786448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b9aa5-4d94-4e51-9de8-afb376d15b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:57.796177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b9aa5-4d94-4e51-9de8-afb376d15b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:57.797634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b9aa5-4d94-4e51-9de8-afb376d15b78 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:57.809136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1128/2248] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:34:58.145543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.146999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:58.160228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59ccae-5f2c-4868-b369-13c0795f192a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.170158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.170648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.170992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.171217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] PATH_INFO: `/v3/projects/f1f35969379f41738710334f3e8438f0` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.172425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f35969379f41738710334f3e8438f0)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.174276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.227426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f35969379f41738710334f3e8438f0', 'request_id': 'req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:58.234513 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f35969379f41738710334f3e8438f0', 'request_id': 'req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:58.240747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f35969379f41738710334f3e8438f0', 'request_id': 'req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:58.255672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] The token cache is being invalidate because project f1f35969379f41738710334f3e8438f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:34:58.255940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:58.256582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f183fd5a-f1bc-4b57-9d98-2ff58a5b1edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:34:58.258277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1121/2249] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:34:58 2022] DELETE /identity/v3/projects/f1f35969379f41738710334f3e8438f0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:34:58.263296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172084f-c310-4c81-811a-394c80bda098 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.263581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172084f-c310-4c81-811a-394c80bda098 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.263863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172084f-c310-4c81-811a-394c80bda098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.285088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b172084f-c310-4c81-811a-394c80bda098 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:58.286360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b172084f-c310-4c81-811a-394c80bda098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:58.305189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:58.363561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1129/2250] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:34:58.371570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a882333-24df-45fc-9917-1ac198b7dc96 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.371852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a882333-24df-45fc-9917-1ac198b7dc96 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.372126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a882333-24df-45fc-9917-1ac198b7dc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.392786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0a882333-24df-45fc-9917-1ac198b7dc96 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:58.394006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0a882333-24df-45fc-9917-1ac198b7dc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:58.402483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1122/2251] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:34:58.406888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172084f-c310-4c81-811a-394c80bda098 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.418906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172084f-c310-4c81-811a-394c80bda098 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.422048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.422673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.422988 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.423262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.423958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:34:58.424261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.425862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.437915 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1130/2252] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:34:58.442266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a882333-24df-45fc-9917-1ac198b7dc96 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.452840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a882333-24df-45fc-9917-1ac198b7dc96 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.457597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.458217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.458497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.458771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.459492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.461388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.490303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1123/2253] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:34:58 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:34:58.495112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.505319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf34a23-0682-4421-b726-6aa8960c3899 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.563775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.564403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.564687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.564962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.565641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.567649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.589676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1131/2254] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:34:58 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:34:58.593918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.605422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb3806-cdf2-4ff8-8ef0-b77c97c1c67a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.610158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.610843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.611221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.611561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.612323 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.613983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.624583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1124/2255] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:34:58.628764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.639193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8920-c9ba-4e13-b74b-c516006049f2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.644100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0378e6a-3b75-4168-92e6-44af62e476dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.644727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0378e6a-3b75-4168-92e6-44af62e476dc admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.645006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0378e6a-3b75-4168-92e6-44af62e476dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.645282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0378e6a-3b75-4168-92e6-44af62e476dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.645995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0378e6a-3b75-4168-92e6-44af62e476dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.647706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0378e6a-3b75-4168-92e6-44af62e476dc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.659222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1132/2256] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:34:58.663266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.673441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93c20c-ab1c-42e6-b8c2-0704aa68dede admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.678043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.678874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.679184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.679462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] PATH_INFO: `/v3/projects/afc49da6dc6343d9ac96dc93df78e7e1/users/783e6935ecc646b4a5d438b9b7de7177/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.699816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc49da6dc6343d9ac96dc93df78e7e1, user_id=783e6935ecc646b4a5d438b9b7de7177, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.702524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.717969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1125/2257] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:34:58 2022] PUT /identity/v3/projects/afc49da6dc6343d9ac96dc93df78e7e1/users/783e6935ecc646b4a5d438b9b7de7177/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:34:58.728531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.728818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.729143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.758770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] MFA Rules not processed for user `783e6935ecc646b4a5d438b9b7de7177`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:34:58.760149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:58.830258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1133/2258] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:34:58.842296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:34:58.847409 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:34:58.871242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:34:58.910345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8e09-fc70-424f-a3e0-c6b679f466fa admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:34:58.914672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:34:58.915336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:34:58.915618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:34:58.915894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:34:58.925445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:34:58.926727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:34:58.936931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1126/2259] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:34:58 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:35:02.789181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:02.794967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:02.813163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:02.872501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9efae0-7557-46ed-802f-ff156d871bef None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:02.878219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:02.878839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:02.879150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:02.879426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:02.881200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:02.921796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:02.924018 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:02.946770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:02.949082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:02.958454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1134/2260] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:35:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:35:02.987660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996dc2c-c9e5-479d-8c6e-b7c15a2421e0 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:02.991292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff1187-020a-4b62-96a4-11bd0db375a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:02.991905 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff1187-020a-4b62-96a4-11bd0db375a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:02.992180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff1187-020a-4b62-96a4-11bd0db375a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:02.992450 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff1187-020a-4b62-96a4-11bd0db375a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:02.994178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8fff1187-020a-4b62-96a4-11bd0db375a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:03.004531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff1187-020a-4b62-96a4-11bd0db375a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:03.005942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff1187-020a-4b62-96a4-11bd0db375a3 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:03.015005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1127/2261] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:35:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:35:10.237918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:10.303119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f9409-d806-4ee0-b7a5-03d19dc3a2ba service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:10.308690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:10.309365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:10.309637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:10.309908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] PATH_INFO: `/v3/users/783e6935ecc646b4a5d438b9b7de7177` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:10.311191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] RBAC: Authorizing `identity:delete_user(user_id=783e6935ecc646b4a5d438b9b7de7177)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:10.313082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:10.368994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783e6935ecc646b4a5d438b9b7de7177'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:10.375922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783e6935ecc646b4a5d438b9b7de7177'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:10.376612 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783e6935ecc646b4a5d438b9b7de7177'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:10.377220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783e6935ecc646b4a5d438b9b7de7177'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:10.378944 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:10.385214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783e6935ecc646b4a5d438b9b7de7177'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:10.403456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1135/2262] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:35:10 2022] DELETE /identity/v3/users/783e6935ecc646b4a5d438b9b7de7177 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:35:10.408285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:10.408795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:10.409055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:10.429584 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:35:10.430762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:10.488225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1128/2263] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:35:10.497741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:10.499187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:10.555207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb60f1-8e8e-40fe-a7ee-aef8cad0084b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:10.559079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:10.559692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:10.559970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:10.560244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:10.569041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:10.570467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:10.579974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1136/2264] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:35:11.068812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:11.070498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:11.083324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7939b-b6ff-4c1a-bc9b-b8ce816fd76d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:11.088127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:11.088771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:11.089116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:11.089396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] PATH_INFO: `/v3/projects/afc49da6dc6343d9ac96dc93df78e7e1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:11.090555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc49da6dc6343d9ac96dc93df78e7e1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:11.092489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:11.140181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc49da6dc6343d9ac96dc93df78e7e1', 'request_id': 'req-74f580b6-410a-44d7-9e58-a7f8007f8b2e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:11.146507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc49da6dc6343d9ac96dc93df78e7e1', 'request_id': 'req-74f580b6-410a-44d7-9e58-a7f8007f8b2e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:11.151479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:11.152771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc49da6dc6343d9ac96dc93df78e7e1', 'request_id': 'req-74f580b6-410a-44d7-9e58-a7f8007f8b2e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:11.164961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] The token cache is being invalidate because project afc49da6dc6343d9ac96dc93df78e7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:35:11.165245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:11.165862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-74f580b6-410a-44d7-9e58-a7f8007f8b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:11.167504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1129/2265] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:35:11 2022] DELETE /identity/v3/projects/afc49da6dc6343d9ac96dc93df78e7e1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:35:13.282516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:13.284504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:13.362151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c5b9-e7e1-4e81-af48-634a7d5cb3f5 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:13.429758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:13.430398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:13.430681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:13.430987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:13.433118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:13.446003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:13.447638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:13.453798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:13.459566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1137/2266] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:35:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:35:21.161758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=3,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:31.847219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:31.847814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:31.847814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:31.866663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:35:31.867891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:31.922218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1130/2267] 10.222.0.17 () {58 vars in 967 bytes} [Sat Aug 6 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:35:31.943862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:31.945240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:32.011127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51522b22-5c12-4ea5-88e0-d00875e7da1e service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:32.016009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:32.016615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:32.016889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:32.017160 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:32.024889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:32.026476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:32.034463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1138/2268] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:35:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:35:41.915678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=25 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:42.031101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=32 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:43.138029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:43.139520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:43.190003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:43.191468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:43.217270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927336f-ccfc-41d9-b8fb-d1ad901c932f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:43.222411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:43.223064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:43.223343 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:43.223629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:43.225799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:43.237081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:43.238640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:43.248972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1131/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:35:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:35:43.259279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe6204-58c5-4ddd-963e-664a6f1518c7 service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:43.264615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:43.265256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:43.265534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:43.265805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] PATH_INFO: `/v3/users/eb6f411ed91946eba54e46799edb12d3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:43.267446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6f411ed91946eba54e46799edb12d3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:43.268910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:43.313680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6f411ed91946eba54e46799edb12d3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.320742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6f411ed91946eba54e46799edb12d3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.321424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6f411ed91946eba54e46799edb12d3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.322023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6f411ed91946eba54e46799edb12d3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.330211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6f411ed91946eba54e46799edb12d3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.349510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1139/2270] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:35:43 2022] DELETE /identity/v3/users/eb6f411ed91946eba54e46799edb12d3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:35:43.523271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:43.533728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a205a-e67c-4b75-987c-0bec2c6e046c service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:43.593653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:43.594288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:43.594561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:43.594832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] PATH_INFO: `/v3/projects/59e7a9fef10f4aba994347a5d714f17f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:43.595996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e7a9fef10f4aba994347a5d714f17f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:43.597871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:43.643285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e7a9fef10f4aba994347a5d714f17f', 'request_id': 'req-d2f751c5-a123-4014-8294-922b61fd2417'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.649254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e7a9fef10f4aba994347a5d714f17f', 'request_id': 'req-d2f751c5-a123-4014-8294-922b61fd2417'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.654801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e7a9fef10f4aba994347a5d714f17f', 'request_id': 'req-d2f751c5-a123-4014-8294-922b61fd2417'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.666965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] The token cache is being invalidate because project 59e7a9fef10f4aba994347a5d714f17f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:35:43.667298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.668407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:35:43.669902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1132/2271] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:35:43 2022] DELETE /identity/v3/projects/59e7a9fef10f4aba994347a5d714f17f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:35:53.345889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:53.663861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:35:54.560997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:35:54.562429 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:54.634511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fdcd3-53c3-4b16-b39e-c91500e7794e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:35:54.695423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c15c2-4115-4706-a6fe-2dd55ded1133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:35:54.696035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c15c2-4115-4706-a6fe-2dd55ded1133 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:35:54.696314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c15c2-4115-4706-a6fe-2dd55ded1133 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:35:54.696587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c15c2-4115-4706-a6fe-2dd55ded1133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:35:54.698441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bc4c15c2-4115-4706-a6fe-2dd55ded1133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:35:54.753699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c15c2-4115-4706-a6fe-2dd55ded1133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:35:54.755329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c15c2-4115-4706-a6fe-2dd55ded1133 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:35:54.764662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1140/2272] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:35:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:36:04.760058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=84 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:36:14.565490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:36:14.566988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:14.583649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f751c5-a123-4014-8294-922b61fd2417 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:36:14.589211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:36:14.589824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:14.590159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:14.590431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:14.592179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:14.651579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:36:14.653675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:36:14.663085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1133/2273] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:36:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:36:22.946772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029b632-b90a-4a9f-a633-647b8fcfa542 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:22.947328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029b632-b90a-4a9f-a633-647b8fcfa542 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:22.947439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029b632-b90a-4a9f-a633-647b8fcfa542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:22.966347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2029b632-b90a-4a9f-a633-647b8fcfa542 None None] MFA Rules not processed for user `4dadf737392d4153ae50de020534f070`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:36:22.967575 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2029b632-b90a-4a9f-a633-647b8fcfa542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:22.978472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1141/2274] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:36:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:36:24.658698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=28 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:36:25.002044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:36:25.003460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:25.017384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d2ae2-6cdf-4ca6-860c-d24ad87d7714 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:36:25.022752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8909d-a76a-486e-b01c-023face033af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:36:25.023383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8909d-a76a-486e-b01c-023face033af service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:25.023661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8909d-a76a-486e-b01c-023face033af service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:25.023929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8909d-a76a-486e-b01c-023face033af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:25.025569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-82d8909d-a76a-486e-b01c-023face033af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:25.082231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8909d-a76a-486e-b01c-023face033af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:36:25.083819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8909d-a76a-486e-b01c-023face033af service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:36:25.092666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1134/2275] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:36:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:36:31.052029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029b632-b90a-4a9f-a633-647b8fcfa542 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:36:31.070541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029b632-b90a-4a9f-a633-647b8fcfa542 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:36:31.077299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:36:31.078153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:31.078534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:31.078973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:31.080946 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:31.096837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:36:31.100117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:36:31.117148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1142/2276] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:36:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:36:35.088547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=28 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:36:40.985647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:40.986261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:40.986261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:41.005363 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] MFA Rules not processed for user `5378c85cd31c42dfbc39ce738a8b6dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:36:41.006555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:41.013524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1135/2277] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:36:41.107736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=9 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:36:41.314947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:36:41.316406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:41.371708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac704a-5d7a-4f16-b13b-b8dc2cd66297 service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:36:41.375266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e246a-a99c-4328-b85f-fc08244cd4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:36:41.375899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e246a-a99c-4328-b85f-fc08244cd4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:41.376176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e246a-a99c-4328-b85f-fc08244cd4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:41.376447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e246a-a99c-4328-b85f-fc08244cd4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:41.378149 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-969e246a-a99c-4328-b85f-fc08244cd4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:41.387734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e246a-a99c-4328-b85f-fc08244cd4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:36:41.389225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e246a-a99c-4328-b85f-fc08244cd4c4 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:36:41.397182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1143/2278] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:36:42.736969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:36:42.738451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:42.825498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4488f-5a04-4b21-9d45-e0fac5ebe704 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:36:42.830619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:36:42.831294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:42.831724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:42.832005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:42.834016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:42.849503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:36:42.850817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:36:42.862462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1136/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:36:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:36:47.240856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a014d2-a70c-4cf7-b5e4-6d2da559323b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:47.241309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a014d2-a70c-4cf7-b5e4-6d2da559323b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:47.241464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a014d2-a70c-4cf7-b5e4-6d2da559323b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:47.260792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-65a014d2-a70c-4cf7-b5e4-6d2da559323b None None] MFA Rules not processed for user `be34f77da4d8407e9a4333bc71f64595`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:36:47.261997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-65a014d2-a70c-4cf7-b5e4-6d2da559323b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:47.316079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1144/2280] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:36:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:36:52.858437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=35 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:36:52.923941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:36:52.938420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628531c-93b7-4384-8e5f-79ee4af27de3 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:36:52.944026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecd54f-c0b6-436d-abba-d5fcdeb28cca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:36:52.944643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecd54f-c0b6-436d-abba-d5fcdeb28cca service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:36:52.944922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecd54f-c0b6-436d-abba-d5fcdeb28cca service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:36:52.945187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecd54f-c0b6-436d-abba-d5fcdeb28cca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:36:52.947027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b8ecd54f-c0b6-436d-abba-d5fcdeb28cca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:36:52.959806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecd54f-c0b6-436d-abba-d5fcdeb28cca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:36:52.961512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecd54f-c0b6-436d-abba-d5fcdeb28cca service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:36:52.972592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1137/2281] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:36:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:36:57.309238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=51 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:01.154781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a014d2-a70c-4cf7-b5e4-6d2da559323b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:01.170208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a014d2-a70c-4cf7-b5e4-6d2da559323b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:01.175793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:01.176472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:01.176818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:01.177159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:01.178932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:01.239185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:01.240852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:01.250781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1145/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:37:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:37:02.967711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:10.136956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:10.137556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:10.137556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:10.157091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:37:10.158289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:10.213184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1138/2283] 10.222.0.17 () {58 vars in 967 bytes} [Sat Aug 6 14:37:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:37:10.262656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:10.264776 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:10.332522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a24e4f-9656-42ec-b396-37b8a7ae42e2 service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:10.337562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:10.338175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:10.338458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:10.338735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:10.347517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:10.349089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:10.357077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1146/2284] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:37:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:37:10.673629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:10.675055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:10.687077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138909c-1aa2-4d45-88e1-8b082c0edade None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:10.690511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:10.691142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:10.691422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:10.691692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:10.693514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:10.753949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:10.755399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:10.765481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1139/2285] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:37:13.898246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:13.900861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:13.922483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1966e-7d2e-40bb-ad05-8d00c4664091 service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:13.928409 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:13.929473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:13.929926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:13.930193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:13.932899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:13.958451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:13.961843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:13.977962 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1147/2286] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:37:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:37:17.118007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:17.119485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:17.132451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986224f6-b12c-4b50-8630-ee8973f0f08a service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:17.137413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:17.138058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:17.138339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:17.138616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] PATH_INFO: `/v3/users/f96728f2ae5d4885858560396971a1b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:17.139817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96728f2ae5d4885858560396971a1b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:17.141713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:17.188262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96728f2ae5d4885858560396971a1b4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.188918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96728f2ae5d4885858560396971a1b4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.189529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96728f2ae5d4885858560396971a1b4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.201547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96728f2ae5d4885858560396971a1b4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.208465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96728f2ae5d4885858560396971a1b4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.221830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1140/2287] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:37:17 2022] DELETE /identity/v3/users/f96728f2ae5d4885858560396971a1b4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:37:17.394620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:17.406103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9d766-3cad-40f0-8398-47cd4a46958d service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:17.462449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:17.463127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:17.463408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:17.463684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] PATH_INFO: `/v3/projects/ca1a30d14e9a4c0ebd2fd8549a09ee44` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:17.464824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1a30d14e9a4c0ebd2fd8549a09ee44)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:17.467263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:17.483523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:17.511316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1a30d14e9a4c0ebd2fd8549a09ee44', 'request_id': 'req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.517180 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1a30d14e9a4c0ebd2fd8549a09ee44', 'request_id': 'req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.524302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1a30d14e9a4c0ebd2fd8549a09ee44', 'request_id': 'req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.535109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] The token cache is being invalidate because project ca1a30d14e9a4c0ebd2fd8549a09ee44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:37:17.535395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.536028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.537532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1148/2288] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:37:17 2022] DELETE /identity/v3/projects/ca1a30d14e9a4c0ebd2fd8549a09ee44 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:37:17.541623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:17.543096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:17.556701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e063-6e47-4b92-a1d8-984c56632320 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:17.564880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:17.615656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:17.616220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:17.616505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:17.616781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] PATH_INFO: `/v3/users/0b066818cbfe40df85381d31a947a1d1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:17.617924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b066818cbfe40df85381d31a947a1d1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:17.619908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:17.664634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b066818cbfe40df85381d31a947a1d1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.665268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b066818cbfe40df85381d31a947a1d1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.665860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b066818cbfe40df85381d31a947a1d1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.677971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b066818cbfe40df85381d31a947a1d1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.688750 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d118fee8-9cf3-439c-888d-9907bf28d2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b066818cbfe40df85381d31a947a1d1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:17.702021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1141/2289] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:37:17 2022] DELETE /identity/v3/users/0b066818cbfe40df85381d31a947a1d1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:37:17.873033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:17.874493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:17.887613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685626b-71cd-4ea8-97c2-ba1c2e7feaf1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:17.944359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:17.945004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:17.945304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:17.945576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] PATH_INFO: `/v3/projects/b9dc80dd9a274082a05cba603b5def7c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:17.947720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9dc80dd9a274082a05cba603b5def7c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:17.949284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:18.000353 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9dc80dd9a274082a05cba603b5def7c', 'request_id': 'req-d642935c-9684-4fd2-8a42-aed169790d9a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:18.006453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9dc80dd9a274082a05cba603b5def7c', 'request_id': 'req-d642935c-9684-4fd2-8a42-aed169790d9a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:18.014906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9dc80dd9a274082a05cba603b5def7c', 'request_id': 'req-d642935c-9684-4fd2-8a42-aed169790d9a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:18.025434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] The token cache is being invalidate because project b9dc80dd9a274082a05cba603b5def7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:37:18.025721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:18.026406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:18.027953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1149/2290] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:37:17 2022] DELETE /identity/v3/projects/b9dc80dd9a274082a05cba603b5def7c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:37:27.698073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:28.022329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:48.000105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97496d4-c6a9-4a29-b8ba-9b0443375b53 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:48.000618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97496d4-c6a9-4a29-b8ba-9b0443375b53 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:48.000725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97496d4-c6a9-4a29-b8ba-9b0443375b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:48.022235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e97496d4-c6a9-4a29-b8ba-9b0443375b53 None None] MFA Rules not processed for user `a3e987dfd35f4a5da36d5034970fcf59`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:37:48.023485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e97496d4-c6a9-4a29-b8ba-9b0443375b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:48.084673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1142/2291] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:37:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:37:48.394191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:48.395601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:48.451601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642935c-9684-4fd2-8a42-aed169790d9a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:48.455161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:48.455769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:48.456046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:48.456315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:48.458016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:48.467012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:48.468422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:48.476147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1150/2292] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:37:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:37:55.608067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97496d4-c6a9-4a29-b8ba-9b0443375b53 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:55.628743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97496d4-c6a9-4a29-b8ba-9b0443375b53 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:55.632628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:55.633243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:55.633521 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:55.633792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:55.635552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:55.753846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:55.755314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:55.775506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1143/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:37:58.473085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=26 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:58.840430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:58.842606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:58.870599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f1b9-135c-4a34-bac2-698791d02697 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:58.876139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:58.876859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:58.877141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:58.877417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] PATH_INFO: `/v3/users/4dadf737392d4153ae50de020534f070` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:58.878590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dadf737392d4153ae50de020534f070)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:58.880578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:58.927608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dadf737392d4153ae50de020534f070'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:58.934696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dadf737392d4153ae50de020534f070'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:58.935568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dadf737392d4153ae50de020534f070'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:58.936039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dadf737392d4153ae50de020534f070'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:58.943206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac8189b2-bd29-459e-9eb6-4ec35cc27d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dadf737392d4153ae50de020534f070'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:58.960715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1151/2294] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:37:58 2022] DELETE /identity/v3/users/4dadf737392d4153ae50de020534f070 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:37:59.135389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.145837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6dfe-9291-4c2d-abaf-4b528b8be8c8 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:59.178686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:59.211558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:59.212296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.212698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.212984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] PATH_INFO: `/v3/projects/d19dc46cdcf549b0a6383c8b99d6f15f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.214328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19dc46cdcf549b0a6383c8b99d6f15f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:59.216282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:59.260901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19dc46cdcf549b0a6383c8b99d6f15f', 'request_id': 'req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:59.267617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19dc46cdcf549b0a6383c8b99d6f15f', 'request_id': 'req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:59.273251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19dc46cdcf549b0a6383c8b99d6f15f', 'request_id': 'req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:59.285506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] The token cache is being invalidate because project d19dc46cdcf549b0a6383c8b99d6f15f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:37:59.285787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:59.286397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-036d1561-cc6a-4d6e-a7c6-dc6bc65d2cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:37:59.287966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1144/2295] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:37:59 2022] DELETE /identity/v3/projects/d19dc46cdcf549b0a6383c8b99d6f15f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:37:59.293866 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946382a4-f2a1-420a-8893-5a1802114054 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.294157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946382a4-f2a1-420a-8893-5a1802114054 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.294438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946382a4-f2a1-420a-8893-5a1802114054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.350772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-946382a4-f2a1-420a-8893-5a1802114054 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:37:59.352010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-946382a4-f2a1-420a-8893-5a1802114054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:59.411217 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1152/2296] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:37:59.419092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74011e7-a3e5-4f44-a0d7-ff94c5b980be None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.419383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74011e7-a3e5-4f44-a0d7-ff94c5b980be None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.419690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74011e7-a3e5-4f44-a0d7-ff94c5b980be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.443318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e74011e7-a3e5-4f44-a0d7-ff94c5b980be None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:37:59.444530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e74011e7-a3e5-4f44-a0d7-ff94c5b980be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:59.453212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1145/2297] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:37:59.457561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946382a4-f2a1-420a-8893-5a1802114054 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.467545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946382a4-f2a1-420a-8893-5a1802114054 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:59.471982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:59.472580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.472856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.473129 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.473816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:37:59.474142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:59.475768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:59.487025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1153/2298] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:37:59.491338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74011e7-a3e5-4f44-a0d7-ff94c5b980be None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.501066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74011e7-a3e5-4f44-a0d7-ff94c5b980be None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:59.505709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:59.506310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.506614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.506908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.507575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:59.509438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:59.536517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1146/2299] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:37:59 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:37:59.540560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.550762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347254c-f751-4fb4-ba01-b9dda20b8cd2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:59.562686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:37:59.615511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:59.616200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.616486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.617039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.617445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:59.619889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:59.644999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1154/2300] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:37:59 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:37:59.646151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.656313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9c5c-6b8a-4020-b7df-3f263bc3bd34 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:59.661235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:59.661940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.662287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.662623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.663455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:59.665078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:59.676187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1147/2301] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:37:59.680821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.692113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71e2bc-a8a4-4be2-b77b-94a0b61ced17 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:59.696902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a57b6f-9991-49bd-9ac0-8035ba453b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:59.697519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a57b6f-9991-49bd-9ac0-8035ba453b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.697792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a57b6f-9991-49bd-9ac0-8035ba453b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.698065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a57b6f-9991-49bd-9ac0-8035ba453b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.698746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a57b6f-9991-49bd-9ac0-8035ba453b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:59.700402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a57b6f-9991-49bd-9ac0-8035ba453b18 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:59.712420 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1155/2302] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:37:59.716756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.727588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccdfc3-8972-4ec8-98a4-8694499dc68d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:37:59.732147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:37:59.733347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.733624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.733901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] PATH_INFO: `/v3/projects/6dd1029bfa7c4713b52e5d63a65ac3ec/users/b63ee5fe10bd43e9a4fe388a02612786/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.752433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd1029bfa7c4713b52e5d63a65ac3ec, user_id=b63ee5fe10bd43e9a4fe388a02612786, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:37:59.755099 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:37:59.769807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1148/2303] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:37:59 2022] PUT /identity/v3/projects/6dd1029bfa7c4713b52e5d63a65ac3ec/users/b63ee5fe10bd43e9a4fe388a02612786/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:37:59.780316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ddae-6064-4393-aaff-5592fb1bab1a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:37:59.780616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ddae-6064-4393-aaff-5592fb1bab1a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:37:59.780910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694ddae-6064-4393-aaff-5592fb1bab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:37:59.810184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f694ddae-6064-4393-aaff-5592fb1bab1a None None] MFA Rules not processed for user `b63ee5fe10bd43e9a4fe388a02612786`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:37:59.811965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f694ddae-6064-4393-aaff-5592fb1bab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:59.877692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1156/2304] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:37:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:37:59.888065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:37:59.890012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:37:59.974473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:00.032484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652ec2-da93-4b37-9cd2-4ec41046f4f2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:00.038809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9298d-1e75-46f6-807b-90aa70192a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:00.039498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9298d-1e75-46f6-807b-90aa70192a48 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:00.039772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9298d-1e75-46f6-807b-90aa70192a48 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:00.040040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9298d-1e75-46f6-807b-90aa70192a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:00.049964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9298d-1e75-46f6-807b-90aa70192a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:00.051611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9298d-1e75-46f6-807b-90aa70192a48 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:00.062668 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1149/2305] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:37:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:02.794002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ddae-6064-4393-aaff-5592fb1bab1a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:02.853714 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694ddae-6064-4393-aaff-5592fb1bab1a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:02.858700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:02.859374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:02.859655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:02.859928 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] PATH_INFO: `/v3/users/b63ee5fe10bd43e9a4fe388a02612786` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:02.861086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63ee5fe10bd43e9a4fe388a02612786)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:02.863025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:02.890380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:02.911235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63ee5fe10bd43e9a4fe388a02612786'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:02.917914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63ee5fe10bd43e9a4fe388a02612786'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:02.918584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63ee5fe10bd43e9a4fe388a02612786'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:02.919282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63ee5fe10bd43e9a4fe388a02612786'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:02.926669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63ee5fe10bd43e9a4fe388a02612786'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:02.943715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1157/2306] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:02 2022] DELETE /identity/v3/users/b63ee5fe10bd43e9a4fe388a02612786 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:02.948181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:02.948676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:02.948774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:02.967352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:02.968533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:03.027350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1150/2307] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:03.037577 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:03.039025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:03.097658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c396-0df7-40cd-ab89-f90c92ead975 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:03.101425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af0b41-56ab-40ec-bd8a-b60eb154a5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:03.102093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0b41-56ab-40ec-bd8a-b60eb154a5be service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.102395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0b41-56ab-40ec-bd8a-b60eb154a5be service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.102654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0b41-56ab-40ec-bd8a-b60eb154a5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.113439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af0b41-56ab-40ec-bd8a-b60eb154a5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:03.114889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af0b41-56ab-40ec-bd8a-b60eb154a5be service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:03.124460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1158/2308] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:03.560975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:03.562394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:03.574466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a022f-ef8e-44bd-ac3a-bc24c35c6f2c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:03.579825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:03.580479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.580858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.581024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] PATH_INFO: `/v3/projects/6dd1029bfa7c4713b52e5d63a65ac3ec` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.582207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd1029bfa7c4713b52e5d63a65ac3ec)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:03.584195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:03.633473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd1029bfa7c4713b52e5d63a65ac3ec', 'request_id': 'req-716d8730-917d-465b-af22-83c9650488e9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:03.640464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd1029bfa7c4713b52e5d63a65ac3ec', 'request_id': 'req-716d8730-917d-465b-af22-83c9650488e9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:03.646169 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd1029bfa7c4713b52e5d63a65ac3ec', 'request_id': 'req-716d8730-917d-465b-af22-83c9650488e9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:03.661657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] The token cache is being invalidate because project 6dd1029bfa7c4713b52e5d63a65ac3ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:03.661941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:03.662627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-716d8730-917d-465b-af22-83c9650488e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:03.664270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1151/2309] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:03 2022] DELETE /identity/v3/projects/6dd1029bfa7c4713b52e5d63a65ac3ec => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:03.669007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333c5d0-bf94-4ea4-af57-7a7848835d07 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.669296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333c5d0-bf94-4ea4-af57-7a7848835d07 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.669570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333c5d0-bf94-4ea4-af57-7a7848835d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.690394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d333c5d0-bf94-4ea4-af57-7a7848835d07 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:03.691657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d333c5d0-bf94-4ea4-af57-7a7848835d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:03.749926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1159/2310] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:03.759721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cebc3-1196-4825-8b6a-5e4b17272433 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.760011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cebc3-1196-4825-8b6a-5e4b17272433 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.760286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cebc3-1196-4825-8b6a-5e4b17272433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.780593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-799cebc3-1196-4825-8b6a-5e4b17272433 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:03.781788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-799cebc3-1196-4825-8b6a-5e4b17272433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:03.789571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1152/2311] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:03.793767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333c5d0-bf94-4ea4-af57-7a7848835d07 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:03.803488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333c5d0-bf94-4ea4-af57-7a7848835d07 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:03.808346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:03.808956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.809237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.809511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.810201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:38:03.811026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:03.812621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:03.823795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1160/2312] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:38:03.828479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cebc3-1196-4825-8b6a-5e4b17272433 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:03.837576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cebc3-1196-4825-8b6a-5e4b17272433 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:03.842293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:03.842930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.843216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.843490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.844152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:03.846050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:03.872418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1153/2313] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:03 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:03.877058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:03.887096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06cd0-193a-469d-af73-11e779921ffd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:03.916656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:03.945329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:03.945939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.946223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.946499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.947187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:03.949110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:03.970362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1161/2314] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:03 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:03.974944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:03.980453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:03.986849 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060a513-f360-402f-8eea-8de3bb365295 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:03.992501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:03.993104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:03.993380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:03.993649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:03.994337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:03.995935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.006174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1154/2315] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:04.009969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.020318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d8b96-586e-46fe-963e-d5ba15edf796 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.024848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684439a7-c31e-472b-b7ad-24f4200d72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.025441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684439a7-c31e-472b-b7ad-24f4200d72f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.025715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684439a7-c31e-472b-b7ad-24f4200d72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.025982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684439a7-c31e-472b-b7ad-24f4200d72f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.026658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684439a7-c31e-472b-b7ad-24f4200d72f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.028282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684439a7-c31e-472b-b7ad-24f4200d72f7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.037918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1162/2316] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:04.042101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.052588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88473f1-a300-47f9-b0ba-079478f310dc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.057464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.058121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.058402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.058673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] PATH_INFO: `/v3/projects/4e4a0646b8f54b8c9df4572f9b0696bc/users/796c50f0d72f43abafefa34472699aa3/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.073766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4a0646b8f54b8c9df4572f9b0696bc, user_id=796c50f0d72f43abafefa34472699aa3, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.076439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.089030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1155/2317] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:04 2022] PUT /identity/v3/projects/4e4a0646b8f54b8c9df4572f9b0696bc/users/796c50f0d72f43abafefa34472699aa3/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:04.100307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803554a-f377-468e-9cbe-466d3da2abfa None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.100716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803554a-f377-468e-9cbe-466d3da2abfa None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.100999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803554a-f377-468e-9cbe-466d3da2abfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.129651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c803554a-f377-468e-9cbe-466d3da2abfa None None] MFA Rules not processed for user `796c50f0d72f43abafefa34472699aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:04.130850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c803554a-f377-468e-9cbe-466d3da2abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:04.194177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1163/2318] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:04.198686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.258323 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d35e70-ef31-4da6-9081-e39729d92589 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.262855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.263618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.263906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.264183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.264841 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.266723 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.292182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1156/2319] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:04 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:04.296933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803554a-f377-468e-9cbe-466d3da2abfa None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.307695 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803554a-f377-468e-9cbe-466d3da2abfa None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.371879 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.372663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.373784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.374072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.374982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.377063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.397964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1164/2320] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:04 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:04.402148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.412808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238dd2b-6db7-4c65-bfe6-7f939d91f1f7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.417383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.417986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.418267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.418541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.419255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.421464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.431696 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1157/2321] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:04.435835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.445523 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489af03-ab78-4b71-923c-ccfdd78d6661 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.450146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.450794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.451106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.451383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] PATH_INFO: `/v3/projects/7435f5e18175409e96e3481c1b061414/users/a31e608733254357a623b61e9f81a82d/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.465550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7435f5e18175409e96e3481c1b061414, user_id=a31e608733254357a623b61e9f81a82d, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.468727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.481607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1165/2322] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:04 2022] PUT /identity/v3/projects/7435f5e18175409e96e3481c1b061414/users/a31e608733254357a623b61e9f81a82d/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:04.486152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.496044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78bec9-b773-498a-8cbb-55356f85ffa4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.551107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8f11d-628a-42d5-b433-fd50b882382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.551714 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8f11d-628a-42d5-b433-fd50b882382e admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.551997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8f11d-628a-42d5-b433-fd50b882382e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.552273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8f11d-628a-42d5-b433-fd50b882382e admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.553035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8f11d-628a-42d5-b433-fd50b882382e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.554617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8f11d-628a-42d5-b433-fd50b882382e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.564564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1158/2323] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:04.568578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.578584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026f02a-74e4-4265-b617-36b78dee3863 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.583389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.584035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.584311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.584578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] PATH_INFO: `/v3/projects/7435f5e18175409e96e3481c1b061414/users/a31e608733254357a623b61e9f81a82d/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.586528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7435f5e18175409e96e3481c1b061414, user_id=a31e608733254357a623b61e9f81a82d, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.589626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.592597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:04.602652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1166/2324] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:04 2022] PUT /identity/v3/projects/7435f5e18175409e96e3481c1b061414/users/a31e608733254357a623b61e9f81a82d/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:04.614717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069308f3-0c40-4311-a4e8-c51fd540110f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.615037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069308f3-0c40-4311-a4e8-c51fd540110f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.615314 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069308f3-0c40-4311-a4e8-c51fd540110f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.642205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-069308f3-0c40-4311-a4e8-c51fd540110f None None] MFA Rules not processed for user `a31e608733254357a623b61e9f81a82d`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:04.643485 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-069308f3-0c40-4311-a4e8-c51fd540110f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:04.670493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:04.712005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1159/2325] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:04.723859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.725565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:04.787377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbae29a-d324-4d9b-bc6e-9e154870fcb2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.793218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.794259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.795292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.795585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.804467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.805889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.815364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1167/2326] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:04.891445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069308f3-0c40-4311-a4e8-c51fd540110f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:04.901036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069308f3-0c40-4311-a4e8-c51fd540110f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:04.905326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc1398-5ab3-4ee8-98fe-27f0d7e22d91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:04.905933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc1398-5ab3-4ee8-98fe-27f0d7e22d91 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:04.906214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc1398-5ab3-4ee8-98fe-27f0d7e22d91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:04.906488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc1398-5ab3-4ee8-98fe-27f0d7e22d91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:04.962631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc1398-5ab3-4ee8-98fe-27f0d7e22d91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:04.964078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc1398-5ab3-4ee8-98fe-27f0d7e22d91 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:04.973281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1160/2327] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:14.811280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:14.969359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=45 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:16.253655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:16.314304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd75b31-6bd6-4426-94df-95dda0783129 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:16.318897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:16.319574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:16.319851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:16.320123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] PATH_INFO: `/v3/users/796c50f0d72f43abafefa34472699aa3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:16.321270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796c50f0d72f43abafefa34472699aa3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:16.323244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:16.367499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796c50f0d72f43abafefa34472699aa3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.374681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796c50f0d72f43abafefa34472699aa3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.375369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796c50f0d72f43abafefa34472699aa3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.375980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796c50f0d72f43abafefa34472699aa3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.383062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796c50f0d72f43abafefa34472699aa3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.400717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1168/2328] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:16 2022] DELETE /identity/v3/users/796c50f0d72f43abafefa34472699aa3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:16.405371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:16.405847 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:16.405965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:16.424237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:16.425433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:16.482210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1161/2329] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:16.492171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:16.493679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:16.571003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179e779-4991-49b2-b6a9-57ad20e9ccb7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:16.574785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:16.575440 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:16.575718 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:16.575991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:16.585482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:16.586922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:16.596253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1169/2330] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:16.862595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:16.864205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:16.879475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61afbe0-442c-45f0-a09e-6ab84cf29986 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:16.884997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:16.885655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:16.885941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:16.886238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] PATH_INFO: `/v3/projects/4e4a0646b8f54b8c9df4572f9b0696bc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:16.887488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4a0646b8f54b8c9df4572f9b0696bc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:16.889423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:16.935426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4a0646b8f54b8c9df4572f9b0696bc', 'request_id': 'req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.941953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4a0646b8f54b8c9df4572f9b0696bc', 'request_id': 'req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.947510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4a0646b8f54b8c9df4572f9b0696bc', 'request_id': 'req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.959828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] The token cache is being invalidate because project 4e4a0646b8f54b8c9df4572f9b0696bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:16.960115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.960733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:16.962265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1162/2331] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:16 2022] DELETE /identity/v3/projects/4e4a0646b8f54b8c9df4572f9b0696bc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:16.966286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:16.971192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:16.977241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9e7e-1a73-4dd5-b8a2-960613c4a95c service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.032639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.033267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.033543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.033813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] PATH_INFO: `/v3/users/a31e608733254357a623b61e9f81a82d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.034978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31e608733254357a623b61e9f81a82d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.036874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.080564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31e608733254357a623b61e9f81a82d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.087192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31e608733254357a623b61e9f81a82d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.087830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31e608733254357a623b61e9f81a82d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.088428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31e608733254357a623b61e9f81a82d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.095479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30ccc0ba-cd20-4e8c-9b8c-6f57f39d2290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31e608733254357a623b61e9f81a82d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.112444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1170/2332] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:16 2022] DELETE /identity/v3/users/a31e608733254357a623b61e9f81a82d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:17.374354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:17.375755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:17.390421 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a912f-1908-4ebe-9e9d-be6ef4f1b196 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.449208 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.449897 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.450117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.450392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] PATH_INFO: `/v3/projects/7435f5e18175409e96e3481c1b061414` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.451546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7435f5e18175409e96e3481c1b061414)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.453437 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.482888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:17.501216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7435f5e18175409e96e3481c1b061414', 'request_id': 'req-36c6c765-0258-4175-a527-f47142315591'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.507554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7435f5e18175409e96e3481c1b061414', 'request_id': 'req-36c6c765-0258-4175-a527-f47142315591'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.513019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7435f5e18175409e96e3481c1b061414', 'request_id': 'req-36c6c765-0258-4175-a527-f47142315591'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.525040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] The token cache is being invalidate because project 7435f5e18175409e96e3481c1b061414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:17.525324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.525953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-36c6c765-0258-4175-a527-f47142315591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:17.527505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1163/2333] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:17 2022] DELETE /identity/v3/projects/7435f5e18175409e96e3481c1b061414 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:17.533023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7e56-5c81-4f76-ac04-9c40e30e09b1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.533513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7e56-5c81-4f76-ac04-9c40e30e09b1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.533613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7e56-5c81-4f76-ac04-9c40e30e09b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.553845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-206a7e56-5c81-4f76-ac04-9c40e30e09b1 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:17.555770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-206a7e56-5c81-4f76-ac04-9c40e30e09b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:17.615542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1171/2334] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:17.623452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f31681-e4ec-4fca-9f77-6cbe5b6cb639 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.623740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f31681-e4ec-4fca-9f77-6cbe5b6cb639 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.624013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f31681-e4ec-4fca-9f77-6cbe5b6cb639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.643387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e5f31681-e4ec-4fca-9f77-6cbe5b6cb639 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:17.644616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e5f31681-e4ec-4fca-9f77-6cbe5b6cb639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:17.652350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1164/2335] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:17.656914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7e56-5c81-4f76-ac04-9c40e30e09b1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:17.666506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7e56-5c81-4f76-ac04-9c40e30e09b1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.671062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.671665 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.671945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.672219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.672935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:38:17.673263 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.674850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.687037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1172/2336] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:38:17.691759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f31681-e4ec-4fca-9f77-6cbe5b6cb639 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:17.700998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f31681-e4ec-4fca-9f77-6cbe5b6cb639 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.706128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.706728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.707033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.707305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.707995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.709862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.736976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1165/2337] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:17 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:38:17.741039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:17.750909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644c98d-b1ba-4363-93c0-9411994e564e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.755480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:17.808467 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.809089 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.809371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.809645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.810311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.812296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.833182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1173/2338] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:17 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:17.837255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:17.848801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a40c03-99e0-49e7-9efe-3f3d17548d55 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.853134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.853777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.854055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.854326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.855053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.856621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.866517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1166/2339] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:17.870740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:17.881235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888dd07-a93f-4ab4-8825-16db5fad8a52 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.885815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a101f3-39f6-440b-976e-50c7e272168e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.886414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a101f3-39f6-440b-976e-50c7e272168e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.886694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a101f3-39f6-440b-976e-50c7e272168e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.887003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a101f3-39f6-440b-976e-50c7e272168e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.887686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a101f3-39f6-440b-976e-50c7e272168e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.889274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a101f3-39f6-440b-976e-50c7e272168e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.900372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1174/2340] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:17.904351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:17.914715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc9b29-1c4c-4ba8-8fad-46be56f8138f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:17.919453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:17.920112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.920391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.920667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] PATH_INFO: `/v3/projects/a970b5817db343d9bf91cff8b29f0583/users/e228c413d7b94997a16d13c9fde626c3/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.935177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a970b5817db343d9bf91cff8b29f0583, user_id=e228c413d7b94997a16d13c9fde626c3, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:17.938272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:17.950693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1167/2341] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:17 2022] PUT /identity/v3/projects/a970b5817db343d9bf91cff8b29f0583/users/e228c413d7b94997a16d13c9fde626c3/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:17.961455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc890-1470-4736-b619-fbd4c294128c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:17.961805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc890-1470-4736-b619-fbd4c294128c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:17.962082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc890-1470-4736-b619-fbd4c294128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:17.988811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a5edc890-1470-4736-b619-fbd4c294128c None None] MFA Rules not processed for user `e228c413d7b94997a16d13c9fde626c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:17.989958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a5edc890-1470-4736-b619-fbd4c294128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:18.053009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1175/2342] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:18.057747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.118522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444c458-879e-4206-b4f7-6c63ce372f9c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.123138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.123747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.124519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.124795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.125458 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.127374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.153017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1168/2343] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:18 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:18.157392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc890-1470-4736-b619-fbd4c294128c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.167992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc890-1470-4736-b619-fbd4c294128c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.223187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.223801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.224079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.224354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.225004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.226927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.247573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1176/2344] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:18 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:18.252050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.262140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a3e61-b908-40a7-b33d-2a35155f2a42 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.266805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.267447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.267727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.268002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.268685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.270258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.280518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1169/2345] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:18.284296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.294136 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab81f9-88f9-4b46-921e-a0a2658fc9d4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.299425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.300073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.300349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.300620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] PATH_INFO: `/v3/projects/ac0d6e132daa4af099ef346b2fbaad10/users/14c19e5e90ab4bd5916d6b409d9784a8/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.302807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:18.317703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0d6e132daa4af099ef346b2fbaad10, user_id=14c19e5e90ab4bd5916d6b409d9784a8, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.320476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.333201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1177/2346] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:18 2022] PUT /identity/v3/projects/ac0d6e132daa4af099ef346b2fbaad10/users/14c19e5e90ab4bd5916d6b409d9784a8/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:18.337168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.346957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246c6ad-5580-4dd5-acb0-00f1d58d8001 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.372532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:18.410222 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56792dcb-026a-4325-8672-fcffcc15ad5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.410840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792dcb-026a-4325-8672-fcffcc15ad5c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.411152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792dcb-026a-4325-8672-fcffcc15ad5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.411423 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56792dcb-026a-4325-8672-fcffcc15ad5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.412095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56792dcb-026a-4325-8672-fcffcc15ad5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.413670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56792dcb-026a-4325-8672-fcffcc15ad5c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.423646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1170/2347] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:18.427747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.438623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5ac8f-fb76-4bb5-8adc-3595bb42ccce admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.443254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.443901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.444179 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.444451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] PATH_INFO: `/v3/projects/ac0d6e132daa4af099ef346b2fbaad10/users/14c19e5e90ab4bd5916d6b409d9784a8/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.446354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0d6e132daa4af099ef346b2fbaad10, user_id=14c19e5e90ab4bd5916d6b409d9784a8, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.448967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.459005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1178/2348] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:18 2022] PUT /identity/v3/projects/ac0d6e132daa4af099ef346b2fbaad10/users/14c19e5e90ab4bd5916d6b409d9784a8/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:18.470052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe388a2b-bfc0-4d76-b7f2-6df8759e35f5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.470541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe388a2b-bfc0-4d76-b7f2-6df8759e35f5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.470655 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe388a2b-bfc0-4d76-b7f2-6df8759e35f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.498700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fe388a2b-bfc0-4d76-b7f2-6df8759e35f5 None None] MFA Rules not processed for user `14c19e5e90ab4bd5916d6b409d9784a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:18.500515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fe388a2b-bfc0-4d76-b7f2-6df8759e35f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:18.565944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1171/2349] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:18.576978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.578705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:18.656912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14465bc-156b-4d40-86a3-db478f87e155 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.660564 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9ff4e-2e01-4a0c-af33-56b429330711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.661174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ff4e-2e01-4a0c-af33-56b429330711 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.661449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ff4e-2e01-4a0c-af33-56b429330711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.661717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9ff4e-2e01-4a0c-af33-56b429330711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.670406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9ff4e-2e01-4a0c-af33-56b429330711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.672462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9ff4e-2e01-4a0c-af33-56b429330711 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.681746 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1179/2350] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:18.745775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe388a2b-bfc0-4d76-b7f2-6df8759e35f5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.806469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe388a2b-bfc0-4d76-b7f2-6df8759e35f5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:18.811267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:18.811902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.812182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.812456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] PATH_INFO: `/v3/users/e228c413d7b94997a16d13c9fde626c3` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.813695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e228c413d7b94997a16d13c9fde626c3)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:18.815667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:18.862736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e228c413d7b94997a16d13c9fde626c3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:18.863416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e228c413d7b94997a16d13c9fde626c3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:18.864021 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e228c413d7b94997a16d13c9fde626c3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:18.865287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:18.877619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e228c413d7b94997a16d13c9fde626c3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:18.884320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e228c413d7b94997a16d13c9fde626c3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:18.897162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1172/2351] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:18 2022] DELETE /identity/v3/users/e228c413d7b94997a16d13c9fde626c3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:18.901633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:18.901918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:18.902219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:18.921710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:18.922932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:18.979280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1180/2352] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:18.989479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:18.990947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:19.046747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59c70d-95c5-44c1-ad57-f0ed127618a1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:19.050420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:19.051063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:19.051346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:19.051620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:19.061269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:19.062629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:19.072027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1173/2353] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:19.335910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:19.337542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:19.354902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69a4af-b90e-4ab2-95ee-6d817c655cec None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:19.359753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:19.360400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:19.360835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:19.361118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] PATH_INFO: `/v3/projects/a970b5817db343d9bf91cff8b29f0583` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:19.362556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a970b5817db343d9bf91cff8b29f0583)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:19.364807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:19.410557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:19.411853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a970b5817db343d9bf91cff8b29f0583', 'request_id': 'req-c63d44e1-2580-4778-9e25-c9b64f60849f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.419735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a970b5817db343d9bf91cff8b29f0583', 'request_id': 'req-c63d44e1-2580-4778-9e25-c9b64f60849f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.426760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a970b5817db343d9bf91cff8b29f0583', 'request_id': 'req-c63d44e1-2580-4778-9e25-c9b64f60849f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.437574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] The token cache is being invalidate because project a970b5817db343d9bf91cff8b29f0583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:19.437861 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.438473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.440011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1181/2354] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:19 2022] DELETE /identity/v3/projects/a970b5817db343d9bf91cff8b29f0583 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:19.443900 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:19.453931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d923cf-894a-4600-90fa-3ce10651113f service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:19.508518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:19.509150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:19.509427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:19.509728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] PATH_INFO: `/v3/users/14c19e5e90ab4bd5916d6b409d9784a8` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:19.510891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c19e5e90ab4bd5916d6b409d9784a8)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:19.512784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:19.557462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c19e5e90ab4bd5916d6b409d9784a8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.558123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c19e5e90ab4bd5916d6b409d9784a8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.558736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c19e5e90ab4bd5916d6b409d9784a8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.570940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c19e5e90ab4bd5916d6b409d9784a8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.575284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:19.580005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c512f369-4328-4163-b70c-69102e117027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c19e5e90ab4bd5916d6b409d9784a8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.593500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1174/2355] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:19 2022] DELETE /identity/v3/users/14c19e5e90ab4bd5916d6b409d9784a8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:19.861123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:19.862620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:19.876997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d44e1-2580-4778-9e25-c9b64f60849f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:19.933620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:19.934259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:19.934533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:19.934803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] PATH_INFO: `/v3/projects/ac0d6e132daa4af099ef346b2fbaad10` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:19.935966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0d6e132daa4af099ef346b2fbaad10)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:19.938484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:19.986013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0d6e132daa4af099ef346b2fbaad10', 'request_id': 'req-af65b94f-e892-492a-9851-f9cfbb317661'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:19.991987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0d6e132daa4af099ef346b2fbaad10', 'request_id': 'req-af65b94f-e892-492a-9851-f9cfbb317661'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:20.000517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0d6e132daa4af099ef346b2fbaad10', 'request_id': 'req-af65b94f-e892-492a-9851-f9cfbb317661'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:20.010828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] The token cache is being invalidate because project ac0d6e132daa4af099ef346b2fbaad10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:20.011137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:20.011756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-af65b94f-e892-492a-9851-f9cfbb317661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:20.013281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1182/2356] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:19 2022] DELETE /identity/v3/projects/ac0d6e132daa4af099ef346b2fbaad10 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:20.018303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40deb12-9a0d-4673-bd3f-af222b001b6e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.018591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40deb12-9a0d-4673-bd3f-af222b001b6e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.018888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40deb12-9a0d-4673-bd3f-af222b001b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.037776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a40deb12-9a0d-4673-bd3f-af222b001b6e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:20.039004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a40deb12-9a0d-4673-bd3f-af222b001b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:20.097320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1175/2357] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:20.105015 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e601bc-7e79-4a33-8c87-91a4f89bf2a8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.105248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e601bc-7e79-4a33-8c87-91a4f89bf2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.105540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e601bc-7e79-4a33-8c87-91a4f89bf2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.125872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-34e601bc-7e79-4a33-8c87-91a4f89bf2a8 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:20.127110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-34e601bc-7e79-4a33-8c87-91a4f89bf2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:20.134895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1183/2358] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:20.139062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40deb12-9a0d-4673-bd3f-af222b001b6e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:20.148962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40deb12-9a0d-4673-bd3f-af222b001b6e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:20.153518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:20.154118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.154392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.154660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.155869 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:38:20.156167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:20.157736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:20.168782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1176/2359] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:38:20.173522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e601bc-7e79-4a33-8c87-91a4f89bf2a8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:20.182837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e601bc-7e79-4a33-8c87-91a4f89bf2a8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:20.187527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:20.188125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.188404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.188676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.189328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:20.191232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:20.217994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1184/2360] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:20 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:20.222142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:20.231889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f888f-95a2-4273-bd92-0ad95f9a710f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:20.287246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:20.287854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.288134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.288406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.289066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:20.290988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:20.312196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1177/2361] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:20 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:38:20.316290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:20.326407 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e60f1-48e9-467f-961b-152cc1b1fbe7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:20.331065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:20.331670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.331949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.332219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.332895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:20.334485 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:20.344851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1185/2362] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:20.348698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:20.359380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b67d8-2292-4a82-a5be-8f85b622d338 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:20.363895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3b423-fc6c-4fb1-a931-4637ca4808f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:20.364493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b423-fc6c-4fb1-a931-4637ca4808f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.364771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b423-fc6c-4fb1-a931-4637ca4808f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.365040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b423-fc6c-4fb1-a931-4637ca4808f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.365718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3b423-fc6c-4fb1-a931-4637ca4808f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:20.367340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3b423-fc6c-4fb1-a931-4637ca4808f8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:20.377555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1178/2363] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:20.381501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:20.391267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75983d-f8d8-4b87-9144-0c7d7689e2b1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:20.395924 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:20.396570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.396851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.397128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] PATH_INFO: `/v3/projects/435a69f3f95144fc84a6a6c237528840/users/d0c65b79a940475f8d8798d35e4d12f7/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.412018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=435a69f3f95144fc84a6a6c237528840, user_id=d0c65b79a940475f8d8798d35e4d12f7, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:20.414642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:20.427586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1186/2364] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:20 2022] PUT /identity/v3/projects/435a69f3f95144fc84a6a6c237528840/users/d0c65b79a940475f8d8798d35e4d12f7/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:20.432717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fb769-c7df-45cf-a679-945dc13e007b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.433005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fb769-c7df-45cf-a679-945dc13e007b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.433277 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fb769-c7df-45cf-a679-945dc13e007b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.451365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e51fb769-c7df-45cf-a679-945dc13e007b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:20.452578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e51fb769-c7df-45cf-a679-945dc13e007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:20.498579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:20.512312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1179/2365] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:20.522337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:20.524038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:20.529296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:20.583045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dbf1f-5238-4bbc-a63e-fa228552fd8e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:20.586657 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:20.587304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:20.587586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:20.587859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:20.597615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:20.599068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:20.608377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1187/2366] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:21.476746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443e4ff-8b3a-4910-93f4-e5082411120a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:21.477036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443e4ff-8b3a-4910-93f4-e5082411120a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:21.477319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443e4ff-8b3a-4910-93f4-e5082411120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:21.508544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c443e4ff-8b3a-4910-93f4-e5082411120a None None] MFA Rules not processed for user `d0c65b79a940475f8d8798d35e4d12f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:21.509779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c443e4ff-8b3a-4910-93f4-e5082411120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:21.578928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1180/2367] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:21.583636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:21.595542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cf7a7-4277-4f22-bbe1-8e7134862ccb service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:21.600836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:21.601455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:21.601737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:21.602012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:21.602676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:21.604723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:21.640231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1188/2368] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:21 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:21.641588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443e4ff-8b3a-4910-93f4-e5082411120a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:21.655923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443e4ff-8b3a-4910-93f4-e5082411120a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:21.715828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:21.716448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:21.716727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:21.717049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:21.718364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:21.720525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:21.741927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1181/2369] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:21 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:21.745948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:21.756983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79e6a6-648a-4cd8-836e-030bdb5a1400 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:21.761674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:21.762286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:21.762568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:21.762837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:21.763555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:21.765157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:21.775875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1189/2370] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:21.781180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:21.791411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc6777-02a6-4bfc-b8f3-44c138943187 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:21.796758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:21.797520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:21.797812 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:21.798106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] PATH_INFO: `/v3/projects/44f566d47ae14b5bbde94fe654139d96/users/d1d9626bfddf48328273e02da4b43555/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:21.813926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f566d47ae14b5bbde94fe654139d96, user_id=d1d9626bfddf48328273e02da4b43555, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:21.816542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:21.829772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1182/2371] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:21 2022] PUT /identity/v3/projects/44f566d47ae14b5bbde94fe654139d96/users/d1d9626bfddf48328273e02da4b43555/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:21.833854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:21.845009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79343d-4c72-42e0-8a02-ccdd750211be admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:21.902834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e0eb2-d573-48a8-8e33-ec774284babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:21.903509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0eb2-d573-48a8-8e33-ec774284babd admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:21.903787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0eb2-d573-48a8-8e33-ec774284babd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:21.904055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0eb2-d573-48a8-8e33-ec774284babd admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:21.904767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e0eb2-d573-48a8-8e33-ec774284babd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:21.907005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e0eb2-d573-48a8-8e33-ec774284babd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:21.917128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1190/2372] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:21.921819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:21.932355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2024f-e31d-49b6-8453-40c4a0837c1d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:21.937268 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:21.937995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:21.938276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:21.938548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] PATH_INFO: `/v3/projects/44f566d47ae14b5bbde94fe654139d96/users/d1d9626bfddf48328273e02da4b43555/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:21.940565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f566d47ae14b5bbde94fe654139d96, user_id=d1d9626bfddf48328273e02da4b43555, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:21.943207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:21.953752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1183/2373] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:21 2022] PUT /identity/v3/projects/44f566d47ae14b5bbde94fe654139d96/users/d1d9626bfddf48328273e02da4b43555/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:22.585148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:22.585640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:22.585760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:22.613455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] MFA Rules not processed for user `d1d9626bfddf48328273e02da4b43555`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:22.614672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:22.675091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:22.687572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1191/2374] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:22.698631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:22.700576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:22.742216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:22.761757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97626-2a33-48e1-b2ee-5df6423eb863 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:22.765882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:22.766525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:22.766811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:22.767121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:22.821875 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:22.823324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:22.832440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1184/2375] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:24.177161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:24.178554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:24.190735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1690c1-34ef-4499-acfb-db00b8a1aaa9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:24.194244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:24.194847 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:24.195166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:24.195437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:24.204294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:24.205716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:24.215007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1192/2376] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:32.828619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=32 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:34.210825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=9 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:38.231165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:38.296291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0a4af-7b51-42da-8e4f-020bda9ea79f service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:38.300887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:38.301525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:38.301808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:38.302107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] PATH_INFO: `/v3/users/d0c65b79a940475f8d8798d35e4d12f7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:38.303297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c65b79a940475f8d8798d35e4d12f7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:38.305215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:38.350496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c65b79a940475f8d8798d35e4d12f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.351176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c65b79a940475f8d8798d35e4d12f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.351787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c65b79a940475f8d8798d35e4d12f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.363669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c65b79a940475f8d8798d35e4d12f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.376046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c65b79a940475f8d8798d35e4d12f7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.392228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1185/2377] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:38 2022] DELETE /identity/v3/users/d0c65b79a940475f8d8798d35e4d12f7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:38.578548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:38.593236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e2b18-59b9-43ee-8df7-5ef7232d19f1 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:38.659170 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:38.659821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:38.660121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:38.660376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] PATH_INFO: `/v3/projects/435a69f3f95144fc84a6a6c237528840` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:38.661581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435a69f3f95144fc84a6a6c237528840)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:38.663531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:38.716583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435a69f3f95144fc84a6a6c237528840', 'request_id': 'req-807c51d0-d6ba-448b-b7f3-af6a749ec501'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.723398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435a69f3f95144fc84a6a6c237528840', 'request_id': 'req-807c51d0-d6ba-448b-b7f3-af6a749ec501'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.732504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435a69f3f95144fc84a6a6c237528840', 'request_id': 'req-807c51d0-d6ba-448b-b7f3-af6a749ec501'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.744610 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] The token cache is being invalidate because project 435a69f3f95144fc84a6a6c237528840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:38.744898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.745538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.747157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1193/2378] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:38 2022] DELETE /identity/v3/projects/435a69f3f95144fc84a6a6c237528840 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:38.751679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:38.753160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:38.766467 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef07748-8c31-4a99-83dd-17cf93b07b80 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:38.832305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:38.833059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:38.833405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:38.833745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] PATH_INFO: `/v3/users/d1d9626bfddf48328273e02da4b43555` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:38.835035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d9626bfddf48328273e02da4b43555)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:38.837042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:38.861463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:38.893927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d9626bfddf48328273e02da4b43555'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.894602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d9626bfddf48328273e02da4b43555'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.895240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d9626bfddf48328273e02da4b43555'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.909418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d9626bfddf48328273e02da4b43555'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.918368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-73d039fe-d441-40df-9093-fd6c7abfe6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d9626bfddf48328273e02da4b43555'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:38.933228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1186/2379] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:38 2022] DELETE /identity/v3/users/d1d9626bfddf48328273e02da4b43555 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:39.107374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.109493 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:39.129732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c51d0-d6ba-448b-b7f3-af6a749ec501 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.184843 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.185484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.185802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.186073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] PATH_INFO: `/v3/projects/44f566d47ae14b5bbde94fe654139d96` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.187244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f566d47ae14b5bbde94fe654139d96)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.189150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.223343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:39.236748 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f566d47ae14b5bbde94fe654139d96', 'request_id': 'req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:39.242802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f566d47ae14b5bbde94fe654139d96', 'request_id': 'req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:39.250074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f566d47ae14b5bbde94fe654139d96', 'request_id': 'req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:39.260548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] The token cache is being invalidate because project 44f566d47ae14b5bbde94fe654139d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:39.260830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:39.261484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ed93ae4e-9057-46b8-ae33-bb2d874e5c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:39.263001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1194/2380] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:39 2022] DELETE /identity/v3/projects/44f566d47ae14b5bbde94fe654139d96 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:39.268364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930b100-6839-49ca-85df-51144e83c9cb None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.268856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930b100-6839-49ca-85df-51144e83c9cb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.268955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930b100-6839-49ca-85df-51144e83c9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.289776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5930b100-6839-49ca-85df-51144e83c9cb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:39.291049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5930b100-6839-49ca-85df-51144e83c9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:39.349466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1187/2381] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:39.358295 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347930d3-d704-418e-9fa8-ae42061fa459 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.358579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347930d3-d704-418e-9fa8-ae42061fa459 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.358850 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347930d3-d704-418e-9fa8-ae42061fa459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.378309 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-347930d3-d704-418e-9fa8-ae42061fa459 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:39.379478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-347930d3-d704-418e-9fa8-ae42061fa459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:39.387256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1195/2382] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:39.392069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930b100-6839-49ca-85df-51144e83c9cb None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.401909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930b100-6839-49ca-85df-51144e83c9cb None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.407238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.407858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.408154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.408400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.409095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:38:39.409400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.411022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.422608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1188/2383] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:38:39.427019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347930d3-d704-418e-9fa8-ae42061fa459 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.436597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347930d3-d704-418e-9fa8-ae42061fa459 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.441297 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.441912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.442191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.442466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.443158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.445063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.472315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1196/2384] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:39 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:39.476445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.486508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946477e3-fde8-46f3-9e53-00286e6bf29f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.541466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.542084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.542366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.542643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.543342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.545248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.566753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1189/2385] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:39 2022] POST /identity/v3/users => generated 281 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:39.570909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.583431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e28d7-4ec8-41a0-9166-b41d74d5dc0f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.588869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.589491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.589797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.590109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.590813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.593031 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.603239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1197/2386] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:39.607544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.618625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a74c26-16c0-4eff-88a6-d7fe53d94ecc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.623931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721d4f8-9bc0-4690-a330-a1401c8884b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.624540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721d4f8-9bc0-4690-a330-a1401c8884b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.624820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721d4f8-9bc0-4690-a330-a1401c8884b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.625095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721d4f8-9bc0-4690-a330-a1401c8884b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.625784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721d4f8-9bc0-4690-a330-a1401c8884b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.627404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721d4f8-9bc0-4690-a330-a1401c8884b6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.641258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1190/2387] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:39.646459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.657097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394858b8-18a7-46a8-9ae3-cd08279d501d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.661706 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.662365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.662642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.662938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] PATH_INFO: `/v3/projects/dbf8343f19fb47ba9ede461a93b21947/users/e5d270f3e77940a68cebdaa05242f82d/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.677132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf8343f19fb47ba9ede461a93b21947, user_id=e5d270f3e77940a68cebdaa05242f82d, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.679828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.693267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1198/2388] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:39 2022] PUT /identity/v3/projects/dbf8343f19fb47ba9ede461a93b21947/users/e5d270f3e77940a68cebdaa05242f82d/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:39.705232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619bf35-d342-4296-bee7-f5c19a2400e8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.705641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619bf35-d342-4296-bee7-f5c19a2400e8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.705918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619bf35-d342-4296-bee7-f5c19a2400e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.714028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:39.734331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4619bf35-d342-4296-bee7-f5c19a2400e8 None None] MFA Rules not processed for user `e5d270f3e77940a68cebdaa05242f82d`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:39.735550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4619bf35-d342-4296-bee7-f5c19a2400e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:39.796677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1191/2389] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:39.830362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:39.833075 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:39.890979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fad25-c168-41b5-b366-fc2ec6f4f75a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:39.894781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f1519-a265-4788-8af2-d310bb945147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:39.895444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1519-a265-4788-8af2-d310bb945147 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:39.895724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1519-a265-4788-8af2-d310bb945147 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:39.895995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1519-a265-4788-8af2-d310bb945147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:39.904282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f1519-a265-4788-8af2-d310bb945147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:39.905694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f1519-a265-4788-8af2-d310bb945147 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:39.914440 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1199/2390] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:39.929081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619bf35-d342-4296-bee7-f5c19a2400e8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:40.004482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619bf35-d342-4296-bee7-f5c19a2400e8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:40.010269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:40.011104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.011533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.011890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] PATH_INFO: `/v3/users/e5d270f3e77940a68cebdaa05242f82d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.013072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d270f3e77940a68cebdaa05242f82d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:40.015134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:40.079711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d270f3e77940a68cebdaa05242f82d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.080519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d270f3e77940a68cebdaa05242f82d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.081134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d270f3e77940a68cebdaa05242f82d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.107145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d270f3e77940a68cebdaa05242f82d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.114721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d270f3e77940a68cebdaa05242f82d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.129890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1192/2391] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:39 2022] DELETE /identity/v3/users/e5d270f3e77940a68cebdaa05242f82d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:40.134552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.134931 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.135277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.154300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:40.155626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:40.205415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:40.215745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1200/2392] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:40.225335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:40.226749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:40.243403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:40.288205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e5fc-ca4a-4d42-b530-9b89180929ba admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:40.291952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bb7d5-17e9-4168-b913-ec6cc362d764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:40.292571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bb7d5-17e9-4168-b913-ec6cc362d764 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.292853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bb7d5-17e9-4168-b913-ec6cc362d764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.293128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bb7d5-17e9-4168-b913-ec6cc362d764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.303680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bb7d5-17e9-4168-b913-ec6cc362d764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:40.306329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bb7d5-17e9-4168-b913-ec6cc362d764 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:40.319371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1193/2393] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:40.589848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:40.591472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:40.605234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffa6f-036d-42b2-a7ff-292bc74c48ea None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:40.610568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:40.611258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.611543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.611825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] PATH_INFO: `/v3/projects/dbf8343f19fb47ba9ede461a93b21947` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.613017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf8343f19fb47ba9ede461a93b21947)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:40.615251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:40.668494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf8343f19fb47ba9ede461a93b21947', 'request_id': 'req-02ac9174-02cf-4121-910a-4642823bde9d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.675192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf8343f19fb47ba9ede461a93b21947', 'request_id': 'req-02ac9174-02cf-4121-910a-4642823bde9d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.682597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf8343f19fb47ba9ede461a93b21947', 'request_id': 'req-02ac9174-02cf-4121-910a-4642823bde9d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.692545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] The token cache is being invalidate because project dbf8343f19fb47ba9ede461a93b21947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:40.692858 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.693472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-02ac9174-02cf-4121-910a-4642823bde9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:40.695092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1201/2394] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:40 2022] DELETE /identity/v3/projects/dbf8343f19fb47ba9ede461a93b21947 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:40.700042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1862d2-3da9-4f06-a367-7ff4f6be8d5e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.700325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1862d2-3da9-4f06-a367-7ff4f6be8d5e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.700599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1862d2-3da9-4f06-a367-7ff4f6be8d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.719879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8a1862d2-3da9-4f06-a367-7ff4f6be8d5e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:40.721121 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8a1862d2-3da9-4f06-a367-7ff4f6be8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:40.778228 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1194/2395] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:40.786532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dacd39-dc67-4bc5-b5df-b1d8998a8a51 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.786941 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dacd39-dc67-4bc5-b5df-b1d8998a8a51 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.787276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dacd39-dc67-4bc5-b5df-b1d8998a8a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.807789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-39dacd39-dc67-4bc5-b5df-b1d8998a8a51 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:40.809001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-39dacd39-dc67-4bc5-b5df-b1d8998a8a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:40.817013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1202/2396] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:40.821371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1862d2-3da9-4f06-a367-7ff4f6be8d5e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:40.831508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1862d2-3da9-4f06-a367-7ff4f6be8d5e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:40.836163 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:40.836773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.837058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.837329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.838023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:38:40.838329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:40.840032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:40.860683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1195/2397] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:38:40.865132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dacd39-dc67-4bc5-b5df-b1d8998a8a51 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:40.874615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dacd39-dc67-4bc5-b5df-b1d8998a8a51 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:40.879590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:40.880273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.880553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.880815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.881477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:40.883454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:40.912183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1203/2398] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:40 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:40.916713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:40.927859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbe9bc-894c-41a8-817d-76f846007c85 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:40.986176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:40.986977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:40.987267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:40.987545 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:40.988213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:40.990133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.011388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1196/2399] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:40 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:41.016359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.026690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f19d6-ff78-4146-8c29-998608a98eb0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.031903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.032520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.032801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.033076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.033768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.035559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.045492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1204/2400] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:38:41.049654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.059912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ac106-879b-46c1-b2a8-1cad65d72d2e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.064659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0f2ac-a3f5-4de8-9f07-dd3b2f3f80bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.065267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f2ac-a3f5-4de8-9f07-dd3b2f3f80bf admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.065551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f2ac-a3f5-4de8-9f07-dd3b2f3f80bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.065848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0f2ac-a3f5-4de8-9f07-dd3b2f3f80bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.066540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0f2ac-a3f5-4de8-9f07-dd3b2f3f80bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.068158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0f2ac-a3f5-4de8-9f07-dd3b2f3f80bf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.080563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1197/2401] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:41.084762 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.095433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343adf5-e2da-4aa8-910c-21021d29fd3f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.100108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.100768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.101052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.101327 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] PATH_INFO: `/v3/projects/ea9ce2191fb9481aa8e733f15cbe6775/users/ffc122b9fbb341129558a733ac5cea64/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.116867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9ce2191fb9481aa8e733f15cbe6775, user_id=ffc122b9fbb341129558a733ac5cea64, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.119699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.134563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1205/2402] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:41 2022] PUT /identity/v3/projects/ea9ce2191fb9481aa8e733f15cbe6775/users/ffc122b9fbb341129558a733ac5cea64/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:41.145351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646edc9b-909a-4df2-848a-a807a8bc906f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.145718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646edc9b-909a-4df2-848a-a807a8bc906f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.146088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646edc9b-909a-4df2-848a-a807a8bc906f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.176907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-646edc9b-909a-4df2-848a-a807a8bc906f None None] MFA Rules not processed for user `ffc122b9fbb341129558a733ac5cea64`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:41.178057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-646edc9b-909a-4df2-848a-a807a8bc906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:41.191907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:41.256095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1198/2403] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:41.260296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.356621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536c127-018f-4699-84df-f27d6681681d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.369440 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.370080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.370366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.370650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.371387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.373833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.401130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1206/2404] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:41 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:41.405232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646edc9b-909a-4df2-848a-a807a8bc906f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.416252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646edc9b-909a-4df2-848a-a807a8bc906f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.479796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.480417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.480693 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.480966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.481635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.484060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.506784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1199/2405] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:41 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:41.510451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.518019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:41.523238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37381190-e0eb-47e0-ad24-fcb9ae43cbdd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.530298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.531050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.531722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.532266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.533195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.534971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.550286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1207/2406] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:41.551522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.563141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec7a2c-3f12-48b2-89d2-c54f3716b07a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.570026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb917a-afda-444e-b175-2860b266ea6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.570899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb917a-afda-444e-b175-2860b266ea6b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.571197 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb917a-afda-444e-b175-2860b266ea6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.571652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb917a-afda-444e-b175-2860b266ea6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.572356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb917a-afda-444e-b175-2860b266ea6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.573942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb917a-afda-444e-b175-2860b266ea6b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.585535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1200/2407] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:41.589768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.600211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a11e1-bd5a-4821-af7f-2e642bd68a49 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.604863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.605516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.605791 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.606061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] PATH_INFO: `/v3/projects/1fba44dd663f4d9895941a807393eb4f/users/3ffce98300334f778a3f95a52a3a4260/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.622538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fba44dd663f4d9895941a807393eb4f, user_id=3ffce98300334f778a3f95a52a3a4260, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.627203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.641325 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1208/2408] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:41 2022] PUT /identity/v3/projects/1fba44dd663f4d9895941a807393eb4f/users/3ffce98300334f778a3f95a52a3a4260/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:41.711413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.711716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.712763 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.746285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] MFA Rules not processed for user `3ffce98300334f778a3f95a52a3a4260`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:41.747554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:41.837938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1201/2409] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:41.842261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.944681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:41.946091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:41.948948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252903a-2b66-4f7f-93aa-7a13a3672b7a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:41.955576 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:41.956144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:41.956424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:41.956697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:41.957377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:41.959986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:41.981043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:41.999806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1209/2410] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:41 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:42.003693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:42.069361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccbf0c-f1d3-4fb2-80c5-e0883a3fae6e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:42.075304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b51b01-f1a3-4da3-9f78-de1436673600 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:42.082976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:42.083745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.084365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.084569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.085483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:42.087606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:42.112764 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1210/2411] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:42 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:38:42.122553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:42.134897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7545dab-8f30-45ff-a66a-ffea55aa1320 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:42.141650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:42.141820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:42.141931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.142050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.142146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.142236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.142324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.142419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.142897 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:42.143846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:42.144730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:42.155253 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1211/2412] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:42.156396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:42.158354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:42.159138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:42.171470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5adf7a-a2ea-4a55-9c46-3fe78a5bb0d3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:42.175265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1202/2413] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:38:42.177090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:42.177743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.178127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.178509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] PATH_INFO: `/v3/projects/e5ad65806e5d427fbb2dd08ce1bbdfa2/users/f7ecfef064194b0c94e78338043ed585/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.195925 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ad65806e5d427fbb2dd08ce1bbdfa2, user_id=f7ecfef064194b0c94e78338043ed585, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:42.198778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:42.213539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1212/2414] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:42 2022] PUT /identity/v3/projects/e5ad65806e5d427fbb2dd08ce1bbdfa2/users/f7ecfef064194b0c94e78338043ed585/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:42.217296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:42.229838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb2818-1e03-435b-b29b-2b433f00a437 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:42.290651 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcdc72-50bc-4965-89b4-f9c35c6085d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:42.291311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcdc72-50bc-4965-89b4-f9c35c6085d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.291604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcdc72-50bc-4965-89b4-f9c35c6085d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.291870 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcdc72-50bc-4965-89b4-f9c35c6085d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.292621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcdc72-50bc-4965-89b4-f9c35c6085d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:42.294202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcdc72-50bc-4965-89b4-f9c35c6085d4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:42.305336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1203/2415] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:42.309541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:42.321663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea894da1-91f1-4724-a79c-9997e7af224d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:42.326310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:42.327105 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.327387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.327662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] PATH_INFO: `/v3/projects/e5ad65806e5d427fbb2dd08ce1bbdfa2/users/f7ecfef064194b0c94e78338043ed585/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.329741 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ad65806e5d427fbb2dd08ce1bbdfa2, user_id=f7ecfef064194b0c94e78338043ed585, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:42.332417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:42.335603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:42.344823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1213/2416] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:42 2022] PUT /identity/v3/projects/e5ad65806e5d427fbb2dd08ce1bbdfa2/users/f7ecfef064194b0c94e78338043ed585/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:42.355040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.355332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.355600 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.386553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] MFA Rules not processed for user `f7ecfef064194b0c94e78338043ed585`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:42.389214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:42.409952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:42.477042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1204/2417] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:42.496073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:42.499077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:42.568951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b56c7f-de6e-4753-92ba-d5dd2f541590 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:42.572854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:42.573508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:42.573783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:42.574054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:42.649751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:42.651382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:42.666490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1214/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:44.194424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:44.195996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:44.211386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8d772-ab60-41f1-b909-3038a5b11d74 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:44.215214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb3c7c-32dc-44e8-aeae-6d70172681d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:44.215830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb3c7c-32dc-44e8-aeae-6d70172681d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:44.216110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb3c7c-32dc-44e8-aeae-6d70172681d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:44.216384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb3c7c-32dc-44e8-aeae-6d70172681d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:44.276079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb3c7c-32dc-44e8-aeae-6d70172681d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:44.277540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb3c7c-32dc-44e8-aeae-6d70172681d3 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:44.288936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1205/2419] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:44.868049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:44.876823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78898-b462-4d35-a5a2-a197520e2d7a service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:44.880662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:44.881522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:44.881804 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:44.882079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:44.892584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:44.894378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:44.908141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1215/2420] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:45.762512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a783-1b53-45c4-b9b2-35aa571164f6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:45.763025 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a783-1b53-45c4-b9b2-35aa571164f6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:45.763149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a783-1b53-45c4-b9b2-35aa571164f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:45.783368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-49a1a783-1b53-45c4-b9b2-35aa571164f6 None None] MFA Rules not processed for user `5378c85cd31c42dfbc39ce738a8b6dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:45.784572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-49a1a783-1b53-45c4-b9b2-35aa571164f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:45.842223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1206/2421] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:49.898552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:49.959251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b00e4-bd42-4129-a185-d9f998e47fe0 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:49.963839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:49.964494 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:49.964765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:49.965041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] PATH_INFO: `/v3/users/ffc122b9fbb341129558a733ac5cea64` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:49.966204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc122b9fbb341129558a733ac5cea64)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:49.968760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:50.009055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:50.015888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc122b9fbb341129558a733ac5cea64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.023028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc122b9fbb341129558a733ac5cea64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.023725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc122b9fbb341129558a733ac5cea64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.024356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc122b9fbb341129558a733ac5cea64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.032681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc122b9fbb341129558a733ac5cea64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.050221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1216/2422] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:49 2022] DELETE /identity/v3/users/ffc122b9fbb341129558a733ac5cea64 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:50.054893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:50.055380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:50.055501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:50.074592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:50.075810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:50.128332 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:50.138047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1207/2423] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:50.148400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:50.149735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:50.206036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecea2d-e158-468b-a068-7c8b34e63bdb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:50.209675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:50.210287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:50.210570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:50.210848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:50.219755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:50.221182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:50.230537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1217/2424] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:50.515070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:50.516629 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:50.530479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23371c-911c-4c1d-9d0a-555074b90e09 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:50.536279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:50.536970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:50.537255 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:50.537527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] PATH_INFO: `/v3/projects/ea9ce2191fb9481aa8e733f15cbe6775` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:50.538677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9ce2191fb9481aa8e733f15cbe6775)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:50.540642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:50.587188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9ce2191fb9481aa8e733f15cbe6775', 'request_id': 'req-1c821a91-c913-4bfb-816b-9eb6e2a2c752'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.593526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9ce2191fb9481aa8e733f15cbe6775', 'request_id': 'req-1c821a91-c913-4bfb-816b-9eb6e2a2c752'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.599493 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9ce2191fb9481aa8e733f15cbe6775', 'request_id': 'req-1c821a91-c913-4bfb-816b-9eb6e2a2c752'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.611965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] The token cache is being invalidate because project ea9ce2191fb9481aa8e733f15cbe6775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:50.612254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.612877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.614377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1208/2425] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:50 2022] DELETE /identity/v3/projects/ea9ce2191fb9481aa8e733f15cbe6775 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:50.618380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:50.628766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245413e-7226-4c0a-8627-2eced85ac7c9 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:50.685134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:50.686304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:50.686570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:50.686841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] PATH_INFO: `/v3/users/3ffce98300334f778a3f95a52a3a4260` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:50.688024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ffce98300334f778a3f95a52a3a4260)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:50.689939 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:50.733869 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ffce98300334f778a3f95a52a3a4260'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.741188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ffce98300334f778a3f95a52a3a4260'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.741903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ffce98300334f778a3f95a52a3a4260'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.742506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ffce98300334f778a3f95a52a3a4260'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.746093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:50.751580 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ffce98300334f778a3f95a52a3a4260'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:50.770705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1218/2426] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:50 2022] DELETE /identity/v3/users/3ffce98300334f778a3f95a52a3a4260 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:51.043063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:51.044573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:51.059057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821a91-c913-4bfb-816b-9eb6e2a2c752 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:51.115372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:51.116005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:51.116282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:51.116553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] PATH_INFO: `/v3/projects/1fba44dd663f4d9895941a807393eb4f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:51.117697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fba44dd663f4d9895941a807393eb4f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:51.119608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:51.165046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fba44dd663f4d9895941a807393eb4f', 'request_id': 'req-c76cac9c-1542-42c5-825d-abb635adf5ab'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.171597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fba44dd663f4d9895941a807393eb4f', 'request_id': 'req-c76cac9c-1542-42c5-825d-abb635adf5ab'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.177055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fba44dd663f4d9895941a807393eb4f', 'request_id': 'req-c76cac9c-1542-42c5-825d-abb635adf5ab'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.190300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] The token cache is being invalidate because project 1fba44dd663f4d9895941a807393eb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:51.190590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.191247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.192789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1209/2427] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:51 2022] DELETE /identity/v3/projects/1fba44dd663f4d9895941a807393eb4f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:51.196867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:51.198395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:51.211323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20702a2a-6187-49c3-a02c-615320d0e983 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:51.269507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:51.270289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:51.270569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:51.270842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] PATH_INFO: `/v3/users/f7ecfef064194b0c94e78338043ed585` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:51.272026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ecfef064194b0c94e78338043ed585)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:51.273935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:51.320893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ecfef064194b0c94e78338043ed585'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.328221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ecfef064194b0c94e78338043ed585'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.328889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ecfef064194b0c94e78338043ed585'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.329502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ecfef064194b0c94e78338043ed585'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.337085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fafbde1a-32dc-4ff8-a125-dfce50497099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ecfef064194b0c94e78338043ed585'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.354531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1219/2428] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:51 2022] DELETE /identity/v3/users/f7ecfef064194b0c94e78338043ed585 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:51.626122 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:51.627613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:51.643229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cac9c-1542-42c5-825d-abb635adf5ab admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:51.651862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:51.709449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:51.710132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:51.710433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:51.710706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] PATH_INFO: `/v3/projects/e5ad65806e5d427fbb2dd08ce1bbdfa2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:51.711966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ad65806e5d427fbb2dd08ce1bbdfa2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:51.713863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:51.760036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ad65806e5d427fbb2dd08ce1bbdfa2', 'request_id': 'req-1e73b9a0-7488-4dc1-9a53-0d069923605c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.766582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ad65806e5d427fbb2dd08ce1bbdfa2', 'request_id': 'req-1e73b9a0-7488-4dc1-9a53-0d069923605c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.772027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ad65806e5d427fbb2dd08ce1bbdfa2', 'request_id': 'req-1e73b9a0-7488-4dc1-9a53-0d069923605c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.784400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] The token cache is being invalidate because project e5ad65806e5d427fbb2dd08ce1bbdfa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:51.784710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.785326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1e73b9a0-7488-4dc1-9a53-0d069923605c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:51.786814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1210/2429] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:51 2022] DELETE /identity/v3/projects/e5ad65806e5d427fbb2dd08ce1bbdfa2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:51.791736 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3f06-685a-4584-bb0e-1d9c94c5cf45 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:51.792025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3f06-685a-4584-bb0e-1d9c94c5cf45 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:51.792304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3f06-685a-4584-bb0e-1d9c94c5cf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:51.812294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-080d3f06-685a-4584-bb0e-1d9c94c5cf45 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:51.813505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-080d3f06-685a-4584-bb0e-1d9c94c5cf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:51.866781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:51.877935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1220/2430] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:51.886827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b03845-ee2f-44df-81ff-c837b2cd494d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:51.887142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b03845-ee2f-44df-81ff-c837b2cd494d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:51.887418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b03845-ee2f-44df-81ff-c837b2cd494d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:51.907792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-42b03845-ee2f-44df-81ff-c837b2cd494d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:51.908998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-42b03845-ee2f-44df-81ff-c837b2cd494d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:51.918603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1211/2431] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:51.921056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d3f06-685a-4584-bb0e-1d9c94c5cf45 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:51.930646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d3f06-685a-4584-bb0e-1d9c94c5cf45 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:51.936063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:51.936669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:51.936943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:51.937211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:51.937898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:38:51.938202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:51.939827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:51.952286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1221/2432] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:38:51.956710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b03845-ee2f-44df-81ff-c837b2cd494d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:51.966431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b03845-ee2f-44df-81ff-c837b2cd494d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:51.971284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:51.971910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:51.972194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:51.972476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:51.973158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:51.975082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:52.011570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1212/2433] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:51 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:52.016076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:52.027379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325f61a-b3ae-463a-afa6-7b4900836e5e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:52.091301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:52.091971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.092255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.092530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.093196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:52.095296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:52.120996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1222/2434] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:52 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:38:52.125868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:52.138050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d1d01-12a9-4be9-9ee2-d78f6ff6c86a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:52.143331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:52.143992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.144376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.145102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.145482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:52.147173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:52.158589 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1213/2435] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:52.162332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:52.174918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327da0b-0cb6-44fc-b67a-a876a96e9685 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:52.181773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0834d8-5615-4775-8ca0-efef1a14a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:52.182745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0834d8-5615-4775-8ca0-efef1a14a154 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.183122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0834d8-5615-4775-8ca0-efef1a14a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.183475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0834d8-5615-4775-8ca0-efef1a14a154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.184238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0834d8-5615-4775-8ca0-efef1a14a154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:52.186513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0834d8-5615-4775-8ca0-efef1a14a154 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:52.198751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1223/2436] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:38:52.202887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:52.214639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661478be-0a0e-4622-878d-43717692a75b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:52.219481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c338c-b4f7-4ed9-af31-538a4007fe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:52.220328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c338c-b4f7-4ed9-af31-538a4007fe70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.220615 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c338c-b4f7-4ed9-af31-538a4007fe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.220898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c338c-b4f7-4ed9-af31-538a4007fe70 admin admin] PATH_INFO: `/v3/projects/b71dd95e6b8f4468b576b1bd7ab4c697/users/767aa349b5d64b078f259a4679217188/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.239059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c338c-b4f7-4ed9-af31-538a4007fe70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71dd95e6b8f4468b576b1bd7ab4c697, user_id=767aa349b5d64b078f259a4679217188, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:52.241058 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c338c-b4f7-4ed9-af31-538a4007fe70 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:52.258340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1214/2437] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:52 2022] PUT /identity/v3/projects/b71dd95e6b8f4468b576b1bd7ab4c697/users/767aa349b5d64b078f259a4679217188/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:52.268617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.268617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.268886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.298107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] MFA Rules not processed for user `767aa349b5d64b078f259a4679217188`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:52.301625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:52.364792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.365372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.365883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.397774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1224/2438] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:52.403828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:52.406360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:52.411539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:52.413669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:52.442389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:52.490210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5684e-af9c-4547-97d4-c701289909cf None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:52.493176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1215/2439] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:38:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:52.498153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:52.498788 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.499107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.499386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.507403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:52.510267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:52.511882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:52.523978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1225/2440] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:38:52.549353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:52.550786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:52.635910 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c83bf-5a8f-434c-9a73-c59a6955ee04 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:52.641229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e104d05-b9a1-41a1-ad99-c6bd3b12baec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:52.641977 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e104d05-b9a1-41a1-ad99-c6bd3b12baec service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:52.642265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e104d05-b9a1-41a1-ad99-c6bd3b12baec service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:52.642541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e104d05-b9a1-41a1-ad99-c6bd3b12baec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:52.653059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e104d05-b9a1-41a1-ad99-c6bd3b12baec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:52.655774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e104d05-b9a1-41a1-ad99-c6bd3b12baec service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:52.664899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1216/2441] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:38:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:38:55.213568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:55.290530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979095b1-17b2-407b-b79c-d9f0b04d9e72 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:55.295334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:55.295997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:55.296276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:55.296556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] PATH_INFO: `/v3/users/767aa349b5d64b078f259a4679217188` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:55.297735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=767aa349b5d64b078f259a4679217188)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:55.299708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:55.395778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767aa349b5d64b078f259a4679217188'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:55.404447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767aa349b5d64b078f259a4679217188'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:55.405233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767aa349b5d64b078f259a4679217188'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:55.405841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767aa349b5d64b078f259a4679217188'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:55.415854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767aa349b5d64b078f259a4679217188'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:55.449508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1226/2442] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:38:55 2022] DELETE /identity/v3/users/767aa349b5d64b078f259a4679217188 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:55.451620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:55.452090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:55.452235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:55.476188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:55.477512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:55.555117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1217/2443] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:55.562922 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:55.564417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:55.647387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0e344-01b4-4e9e-9316-8d4afedbaca8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:55.667793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0912b-9659-48b2-81a9-4ae1c1ec1925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:55.670368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0912b-9659-48b2-81a9-4ae1c1ec1925 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:55.670652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0912b-9659-48b2-81a9-4ae1c1ec1925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:55.671018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0912b-9659-48b2-81a9-4ae1c1ec1925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:55.684446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0912b-9659-48b2-81a9-4ae1c1ec1925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:55.686300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0912b-9659-48b2-81a9-4ae1c1ec1925 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:55.715184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1227/2444] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:38:56.251090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:56.253800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:56.269198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a71ae-8424-42d4-ae0c-c250e8de8a85 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:56.276329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:56.277275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.277911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.278434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] PATH_INFO: `/v3/projects/b71dd95e6b8f4468b576b1bd7ab4c697` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.279825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71dd95e6b8f4468b576b1bd7ab4c697)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:56.281955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:56.342275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71dd95e6b8f4468b576b1bd7ab4c697', 'request_id': 'req-8a1dbcba-2ec0-404e-bb13-525cecd7f365'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:56.349996 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71dd95e6b8f4468b576b1bd7ab4c697', 'request_id': 'req-8a1dbcba-2ec0-404e-bb13-525cecd7f365'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:56.354541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:56.356770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71dd95e6b8f4468b576b1bd7ab4c697', 'request_id': 'req-8a1dbcba-2ec0-404e-bb13-525cecd7f365'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:56.373818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] The token cache is being invalidate because project b71dd95e6b8f4468b576b1bd7ab4c697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:38:56.374107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:56.374976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8a1dbcba-2ec0-404e-bb13-525cecd7f365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:38:56.376999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1218/2445] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:38:56 2022] DELETE /identity/v3/projects/b71dd95e6b8f4468b576b1bd7ab4c697 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:38:56.384866 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70a6c-011c-413b-923b-03817600da72 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.385405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70a6c-011c-413b-923b-03817600da72 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.385537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70a6c-011c-413b-923b-03817600da72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.407626 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1aa70a6c-011c-413b-923b-03817600da72 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:56.408860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1aa70a6c-011c-413b-923b-03817600da72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:56.423268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:38:56.487503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1228/2446] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:38:56.489245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4e3fc-2e7e-4bf6-a787-9311203ad662 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.489694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4e3fc-2e7e-4bf6-a787-9311203ad662 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.489779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4e3fc-2e7e-4bf6-a787-9311203ad662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.515907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e2c4e3fc-2e7e-4bf6-a787-9311203ad662 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:56.517358 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e2c4e3fc-2e7e-4bf6-a787-9311203ad662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:56.527422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1219/2447] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:56.531926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70a6c-011c-413b-923b-03817600da72 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:56.542559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70a6c-011c-413b-923b-03817600da72 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:56.550038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:56.550664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.550983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.551264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.551967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:38:56.552273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:56.553912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:56.567422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1229/2448] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:38:56.572641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4e3fc-2e7e-4bf6-a787-9311203ad662 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:56.588512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4e3fc-2e7e-4bf6-a787-9311203ad662 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:56.596427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:56.597067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.597349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.597623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.598566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:56.600965 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:56.635222 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1220/2449] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:38:56 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:38:56.641208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:56.654644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dad11-cd66-4fc8-9108-2dd9deeb10e3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:56.717888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:56.718512 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.718802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.719863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.719863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:56.721683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:56.746086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1230/2450] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:38:56 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:38:56.750490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:56.761853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b2a59-9815-42bd-ac04-c21e2ed8c2be admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:56.767542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:56.768361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.768840 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.769108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.769860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:56.771621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:56.785578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1221/2451] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:56.787109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:56.799734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e1f10-ee19-4b1f-8bfd-833d58bc89c0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:56.808211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aca5c-74d4-43b9-9db9-40fdac878703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:56.808838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aca5c-74d4-43b9-9db9-40fdac878703 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.809178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aca5c-74d4-43b9-9db9-40fdac878703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.809471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aca5c-74d4-43b9-9db9-40fdac878703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.810245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aca5c-74d4-43b9-9db9-40fdac878703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:56.812216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aca5c-74d4-43b9-9db9-40fdac878703 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:56.825266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1231/2452] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:38:56.830024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:56.840583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8428bcf-4bf4-440f-9544-6dc6f1ca955c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:56.851767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:56.851767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.851767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.851767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] PATH_INFO: `/v3/projects/f729ce30d3cc4915ad65f03c370eef00/users/d4c12d90abaf4fbfa422242f55c847bc/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.866475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f729ce30d3cc4915ad65f03c370eef00, user_id=d4c12d90abaf4fbfa422242f55c847bc, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:56.869269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:56.883325 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1222/2453] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:38:56 2022] PUT /identity/v3/projects/f729ce30d3cc4915ad65f03c370eef00/users/d4c12d90abaf4fbfa422242f55c847bc/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:38:56.895294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9900cb-40b6-455d-9780-566b6bab8db2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:56.895948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9900cb-40b6-455d-9780-566b6bab8db2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:56.896065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9900cb-40b6-455d-9780-566b6bab8db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:56.925516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cf9900cb-40b6-455d-9780-566b6bab8db2 None None] MFA Rules not processed for user `d4c12d90abaf4fbfa422242f55c847bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:38:56.926725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cf9900cb-40b6-455d-9780-566b6bab8db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:57.000404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1232/2454] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:38:57.010700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:38:57.012566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:38:57.094149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e102eb3-e6e4-4d12-8154-e9739148367d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:38:57.098344 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb55f2-4db8-4d40-bcaf-32728f07f36e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:38:57.099003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb55f2-4db8-4d40-bcaf-32728f07f36e service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:38:57.099339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb55f2-4db8-4d40-bcaf-32728f07f36e service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:38:57.099568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb55f2-4db8-4d40-bcaf-32728f07f36e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:38:57.109922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb55f2-4db8-4d40-bcaf-32728f07f36e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:38:57.111426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb55f2-4db8-4d40-bcaf-32728f07f36e service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:38:57.123590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1223/2455] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:38:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:39:06.996372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:39:07.118087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=2,keystone:SELECT=56,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:39:15.045589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a438ab-9c61-4656-8c38-a69f0f5ab5c0 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.046236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a438ab-9c61-4656-8c38-a69f0f5ab5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.046236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a438ab-9c61-4656-8c38-a69f0f5ab5c0 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.047702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1233/2456] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:39:15.053989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb3730-e7ed-438f-b513-335a56cddd87 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.054475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb3730-e7ed-438f-b513-335a56cddd87 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.054674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb3730-e7ed-438f-b513-335a56cddd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.075452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-dccb3730-e7ed-438f-b513-335a56cddd87 None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:39:15.076641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dccb3730-e7ed-438f-b513-335a56cddd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:15.179533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1224/2457] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:39:15.193074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a438ab-9c61-4656-8c38-a69f0f5ab5c0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:15.195370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-56a438ab-9c61-4656-8c38-a69f0f5ab5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:15.258276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a438ab-9c61-4656-8c38-a69f0f5ab5c0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:15.261845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1286fa-1696-457a-91a1-6948a71d114b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:15.262459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1286fa-1696-457a-91a1-6948a71d114b service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.263361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1286fa-1696-457a-91a1-6948a71d114b service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.263682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1286fa-1696-457a-91a1-6948a71d114b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.273937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1286fa-1696-457a-91a1-6948a71d114b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:15.275422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1286fa-1696-457a-91a1-6948a71d114b service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:15.286712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1234/2458] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:39:15.354126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e738-c379-4adf-8329-a2dd8215a287 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.354412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e738-c379-4adf-8329-a2dd8215a287 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.354689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e738-c379-4adf-8329-a2dd8215a287 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.356084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1225/2459] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:39:15.360814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281ea1a-3bd3-44db-ba95-d81c81554f48 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.361107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281ea1a-3bd3-44db-ba95-d81c81554f48 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.361387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281ea1a-3bd3-44db-ba95-d81c81554f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.381035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8281ea1a-3bd3-44db-ba95-d81c81554f48 None None] MFA Rules not processed for user `cd235744a7ee462eb13e383930de4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:39:15.382231 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8281ea1a-3bd3-44db-ba95-d81c81554f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:15.431046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1235/2460] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:39:15.442853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e738-c379-4adf-8329-a2dd8215a287 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:15.444553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3070e738-c379-4adf-8329-a2dd8215a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:15.457961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e738-c379-4adf-8329-a2dd8215a287 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:15.461504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbb2b7-1187-4b2a-b64a-2c41e34fbfd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:15.462105 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb2b7-1187-4b2a-b64a-2c41e34fbfd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.462383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb2b7-1187-4b2a-b64a-2c41e34fbfd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.462661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb2b7-1187-4b2a-b64a-2c41e34fbfd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.470284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbb2b7-1187-4b2a-b64a-2c41e34fbfd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:15.471771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbb2b7-1187-4b2a-b64a-2c41e34fbfd3 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:15.479657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1226/2461] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:39:15.563750 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f255217-3944-4ab0-b7ba-958d37c18823 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.564037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f255217-3944-4ab0-b7ba-958d37c18823 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.564313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f255217-3944-4ab0-b7ba-958d37c18823 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.565672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1236/2462] 10.222.0.17 () {54 vars in 921 bytes} [Sat Aug 6 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 14:39:15.570885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1e42-3b99-4eba-8b33-351772b9ea7d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.571176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1e42-3b99-4eba-8b33-351772b9ea7d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.571448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1e42-3b99-4eba-8b33-351772b9ea7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.592540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-820a1e42-3b99-4eba-8b33-351772b9ea7d None None] MFA Rules not processed for user `075e9c0ef8fd406087d05a6ce36dc4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:39:15.594040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-820a1e42-3b99-4eba-8b33-351772b9ea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:15.606858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1227/2463] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Aug 6 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:39:15.619332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f255217-3944-4ab0-b7ba-958d37c18823 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:15.633999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f255217-3944-4ab0-b7ba-958d37c18823 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:15.639894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:15.640529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:15.640825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:15.641106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:15.653121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:15.661645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:15.674356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1237/2464] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:39:17.463197 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac6209-4b35-4ee2-be04-04f6d9e83b87 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:17.463671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac6209-4b35-4ee2-be04-04f6d9e83b87 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:17.463865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac6209-4b35-4ee2-be04-04f6d9e83b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:17.484096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-59ac6209-4b35-4ee2-be04-04f6d9e83b87 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:39:17.485516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-59ac6209-4b35-4ee2-be04-04f6d9e83b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:17.551223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1228/2465] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:39:17.568419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:17.569983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:17.641390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af34627-f2e2-4c35-b671-6bf96774c912 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:17.646457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd86711-d0c0-4cbe-9151-2aee3f044049 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:17.647101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd86711-d0c0-4cbe-9151-2aee3f044049 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:17.647387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd86711-d0c0-4cbe-9151-2aee3f044049 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:17.647662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd86711-d0c0-4cbe-9151-2aee3f044049 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:17.655720 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd86711-d0c0-4cbe-9151-2aee3f044049 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:17.657299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd86711-d0c0-4cbe-9151-2aee3f044049 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:17.665345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1238/2466] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:39:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:39:20.850751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac6209-4b35-4ee2-be04-04f6d9e83b87 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:20.917982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac6209-4b35-4ee2-be04-04f6d9e83b87 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:20.923094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:20.923736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:20.924014 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:20.924288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] PATH_INFO: `/v3/users/d4c12d90abaf4fbfa422242f55c847bc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:20.925456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c12d90abaf4fbfa422242f55c847bc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:20.927426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:20.953755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:39:20.982253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c12d90abaf4fbfa422242f55c847bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:20.983377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c12d90abaf4fbfa422242f55c847bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:20.984194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c12d90abaf4fbfa422242f55c847bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:20.997654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c12d90abaf4fbfa422242f55c847bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:21.005199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c12d90abaf4fbfa422242f55c847bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:21.024742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1229/2467] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:39:20 2022] DELETE /identity/v3/users/d4c12d90abaf4fbfa422242f55c847bc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:39:21.025874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:21.026355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:21.026355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:21.059728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:39:21.060986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:21.099038 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:39:21.163023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1239/2468] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:39:21.175465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:21.183688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:21.258457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50046fdd-61d7-4092-aa6e-a5941dc61c04 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:21.262140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeae6b5-ec05-4057-85ee-6a5f50a180cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:21.264370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeae6b5-ec05-4057-85ee-6a5f50a180cf service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:21.264648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeae6b5-ec05-4057-85ee-6a5f50a180cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:21.264917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeae6b5-ec05-4057-85ee-6a5f50a180cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:21.274506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeae6b5-ec05-4057-85ee-6a5f50a180cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:21.275964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeae6b5-ec05-4057-85ee-6a5f50a180cf service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:21.294460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1230/2469] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:39:21.587137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:21.588908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:21.611400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee560b84-f99b-49a3-9525-9e74ae283f4b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:21.616775 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:21.617504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:21.618516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:21.618812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] PATH_INFO: `/v3/projects/f729ce30d3cc4915ad65f03c370eef00` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:21.620308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f729ce30d3cc4915ad65f03c370eef00)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:21.622832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:21.677471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f729ce30d3cc4915ad65f03c370eef00', 'request_id': 'req-9a211db0-8cc9-4aac-adcb-588fc26055e2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:21.684163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f729ce30d3cc4915ad65f03c370eef00', 'request_id': 'req-9a211db0-8cc9-4aac-adcb-588fc26055e2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:21.693293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f729ce30d3cc4915ad65f03c370eef00', 'request_id': 'req-9a211db0-8cc9-4aac-adcb-588fc26055e2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:21.704908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] The token cache is being invalidate because project f729ce30d3cc4915ad65f03c370eef00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:39:21.705216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:21.705868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:39:21.707677 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1240/2470] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:39:21 2022] DELETE /identity/v3/projects/f729ce30d3cc4915ad65f03c370eef00 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:39:31.283181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=8,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:39:31.701508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:39:39.356688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:39.357143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:39.357239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:39.376273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:39:39.377446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:39.430816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1231/2471] 10.222.0.17 () {58 vars in 967 bytes} [Sat Aug 6 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:39:39.443257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:39.444652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:39.511221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a211db0-8cc9-4aac-adcb-588fc26055e2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:39.516163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:39.516774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:39.517050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:39.517357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:39.525177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:39.526756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:39.535282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1241/2472] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:39:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:39:48.895982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:48.897361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:48.953296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789248-9bd9-4be2-9689-8941acc987e0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:48.956856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:48.957470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:48.957752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:48.958029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:48.959740 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:49.028292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:49.029793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:49.040494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1232/2473] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:39:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:39:49.531910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=32 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:39:58.251417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:39:58.252811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:58.264425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ea19a-1974-4cfc-ae22-4d7c8f39d709 service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:39:58.267933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:39:58.268530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:39:58.268807 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:39:58.269077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:39:58.270713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:39:58.283900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:39:58.285308 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:39:58.295952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1242/2474] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:39:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:39:59.035880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=79 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:05.994333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:05.995929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:06.054529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6731773-ce0e-4a6d-a1d9-4a2cc4c469c8 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:06.058440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:06.059089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:06.059369 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:06.059643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:06.061459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:06.072232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:06.073787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:06.082384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1233/2475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:40:08.291215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:13.495735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:13.498206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:13.513050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4ac7-1f0a-43ec-8d5d-fc2b3a120a08 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:13.519092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:13.519725 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:13.520004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:13.520495 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:13.522395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:13.537221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:13.538890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:13.564673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1243/2476] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:40:16.078383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=26 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:19.304695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:19.313552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5c21b-3534-45b8-8a9e-723591dd9cf9 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:19.317196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:19.317798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:19.318081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:19.318354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:19.320204 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:19.383576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:19.384984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:19.395226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1234/2477] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:40:22.251863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:22.253519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:22.269435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb00da-5d0a-4d36-a6bf-95d46ce0989f service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:22.274433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:22.275529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:22.276125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:22.276218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] PATH_INFO: `/v3/users/a3e987dfd35f4a5da36d5034970fcf59` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:22.292598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e987dfd35f4a5da36d5034970fcf59)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:22.294567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:22.345296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e987dfd35f4a5da36d5034970fcf59'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.352451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e987dfd35f4a5da36d5034970fcf59'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.353159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e987dfd35f4a5da36d5034970fcf59'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.353761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e987dfd35f4a5da36d5034970fcf59'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.361470 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69236d0-4612-4d35-97a4-4abab0a1fdaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e987dfd35f4a5da36d5034970fcf59'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.381499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1244/2478] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:40:22 2022] DELETE /identity/v3/users/a3e987dfd35f4a5da36d5034970fcf59 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:22.555460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:22.567198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982b1ef-4346-45c5-840c-875d1517c0c1 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:22.625707 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:22.626274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:22.626559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:22.626828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] PATH_INFO: `/v3/projects/c17d83661330425d8c75a2562df8e16e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:22.637045 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c17d83661330425d8c75a2562df8e16e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:22.639530 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:22.685089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c17d83661330425d8c75a2562df8e16e', 'request_id': 'req-f04439f0-f1b3-49c3-a549-2a1da099d143'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.692056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c17d83661330425d8c75a2562df8e16e', 'request_id': 'req-f04439f0-f1b3-49c3-a549-2a1da099d143'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.697446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c17d83661330425d8c75a2562df8e16e', 'request_id': 'req-f04439f0-f1b3-49c3-a549-2a1da099d143'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.710009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] The token cache is being invalidate because project c17d83661330425d8c75a2562df8e16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:40:22.710294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.710944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f04439f0-f1b3-49c3-a549-2a1da099d143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:22.712438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1235/2479] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:40:22 2022] DELETE /identity/v3/projects/c17d83661330425d8c75a2562df8e16e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:22.717402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aad666-a15d-418a-bc96-91b65ade1075 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:22.717696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aad666-a15d-418a-bc96-91b65ade1075 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:22.717978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aad666-a15d-418a-bc96-91b65ade1075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:22.737202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-84aad666-a15d-418a-bc96-91b65ade1075 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:22.738395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-84aad666-a15d-418a-bc96-91b65ade1075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:22.796064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1245/2480] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:40:22.804069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ebc70-f9dd-477f-9623-3d655192773b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:22.804411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ebc70-f9dd-477f-9623-3d655192773b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:22.804789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ebc70-f9dd-477f-9623-3d655192773b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:22.825959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3b0ebc70-f9dd-477f-9623-3d655192773b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:22.827822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3b0ebc70-f9dd-477f-9623-3d655192773b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:22.835630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1236/2481] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:40:22.839866 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aad666-a15d-418a-bc96-91b65ade1075 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:22.849530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aad666-a15d-418a-bc96-91b65ade1075 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:22.854162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:22.854763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:22.855074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:22.855347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:22.856035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:40:22.856337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:22.857917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:22.869388 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1246/2482] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:40:22.874116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ebc70-f9dd-477f-9623-3d655192773b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:22.883190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ebc70-f9dd-477f-9623-3d655192773b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:22.887756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:22.888350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:22.888630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:22.888899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:22.890103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:22.892001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:22.896985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:22.928758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1237/2483] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:40:22 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:40:22.933210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:22.943841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889911aa-da55-4bf5-bb8b-5c67bf6fcc9f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:22.975603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:23.001486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.002097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.002380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.002654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.003365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.005268 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.026161 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1247/2484] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:40:22 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:40:23.030596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.040894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a6f23-f40e-414b-9c99-b26830962cf9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.046075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.046679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.046995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.047275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.047963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.049544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.059961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1238/2485] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:23.064302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.074096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c43f0-0dc1-427c-9c78-257bf4bc4669 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.078622 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f29c0-eb0c-4fc3-8481-3f7680d1fdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.079251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f29c0-eb0c-4fc3-8481-3f7680d1fdfc admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.079530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f29c0-eb0c-4fc3-8481-3f7680d1fdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.079801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f29c0-eb0c-4fc3-8481-3f7680d1fdfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.080478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f29c0-eb0c-4fc3-8481-3f7680d1fdfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.082070 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f29c0-eb0c-4fc3-8481-3f7680d1fdfc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.091645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1248/2486] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:23.095899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.105972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7aae2-114d-4b45-932e-19755b80ab62 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.111286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.111943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.112224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.112499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] PATH_INFO: `/v3/projects/3f51a9db8177444e8e69f8954692204e/users/726bbf45500c43db83300419f6390769/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.127474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f51a9db8177444e8e69f8954692204e, user_id=726bbf45500c43db83300419f6390769, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.130072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.143466 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1239/2487] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:40:23 2022] PUT /identity/v3/projects/3f51a9db8177444e8e69f8954692204e/users/726bbf45500c43db83300419f6390769/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:23.155026 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847002bd-dd4d-4959-a690-2c34a6b4def6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.155318 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847002bd-dd4d-4959-a690-2c34a6b4def6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.155600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847002bd-dd4d-4959-a690-2c34a6b4def6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.185072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-847002bd-dd4d-4959-a690-2c34a6b4def6 None None] MFA Rules not processed for user `726bbf45500c43db83300419f6390769`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:23.187778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-847002bd-dd4d-4959-a690-2c34a6b4def6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:23.253369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1249/2488] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:40:23.257934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.320115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b8b04-a3ae-4cb9-afb4-19aae9eca91d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.324766 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.325380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.325660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.325951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.326650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.328577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.354966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1240/2489] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:40:23 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:40:23.359668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847002bd-dd4d-4959-a690-2c34a6b4def6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.370229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847002bd-dd4d-4959-a690-2c34a6b4def6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.427480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.428108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.428389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.428663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.429425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.431392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.453139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1250/2490] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:40:23 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:40:23.457534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.468414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cd0e5-8c65-4e1f-bf11-336aedc50ebb admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.473024 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.473912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.474195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.474465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.475177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.476756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.487971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1241/2491] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:23.492079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.502005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785068c6-218a-436a-935f-a7644d55f851 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.506569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.507255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.507532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.507803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] PATH_INFO: `/v3/projects/eaeebc93abfb462b86fd6b64291d78a7/users/85e767005a2e4158b004bf85270b1689/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.650937 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaeebc93abfb462b86fd6b64291d78a7, user_id=85e767005a2e4158b004bf85270b1689, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.653581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.667059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1251/2492] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:40:23 2022] PUT /identity/v3/projects/eaeebc93abfb462b86fd6b64291d78a7/users/85e767005a2e4158b004bf85270b1689/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:23.671712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.682284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71e8e-176c-45c6-a53b-d6857669ffab admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.740815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc54b6d-b612-46d7-bea8-dfb21b539d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.741549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc54b6d-b612-46d7-bea8-dfb21b539d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.741855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc54b6d-b612-46d7-bea8-dfb21b539d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.742127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc54b6d-b612-46d7-bea8-dfb21b539d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.742811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc54b6d-b612-46d7-bea8-dfb21b539d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.744419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc54b6d-b612-46d7-bea8-dfb21b539d30 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.754949 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1242/2493] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:23.759250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.769462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e327bf-ba50-4019-a437-68f663a4314e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.774077 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.774730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.775048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.775320 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] PATH_INFO: `/v3/projects/eaeebc93abfb462b86fd6b64291d78a7/users/85e767005a2e4158b004bf85270b1689/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.777266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaeebc93abfb462b86fd6b64291d78a7, user_id=85e767005a2e4158b004bf85270b1689, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.780518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:23.783646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:23.792438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1252/2494] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:40:23 2022] PUT /identity/v3/projects/eaeebc93abfb462b86fd6b64291d78a7/users/85e767005a2e4158b004bf85270b1689/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:23.803404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffec19-6cd4-4695-bc27-7bf39c8ffcd9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.803691 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffec19-6cd4-4695-bc27-7bf39c8ffcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.803964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffec19-6cd4-4695-bc27-7bf39c8ffcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.830028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:23.834136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5fffec19-6cd4-4695-bc27-7bf39c8ffcd9 None None] MFA Rules not processed for user `85e767005a2e4158b004bf85270b1689`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:23.836412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5fffec19-6cd4-4695-bc27-7bf39c8ffcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:23.911367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1243/2495] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:40:23.922479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:23.924203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:23.980403 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912790c-bdd4-4c64-b2d3-1f69fc4a3a28 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:23.984043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:23.984659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:23.984938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:23.985214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:23.994037 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:23.995492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:24.004663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1253/2496] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:40:24.080606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffec19-6cd4-4695-bc27-7bf39c8ffcd9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:24.090097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffec19-6cd4-4695-bc27-7bf39c8ffcd9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:24.094293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:24.094921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:24.095206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:24.095482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:24.151506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:24.152928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:24.161765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1244/2497] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:40:34.000611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=28 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:34.157710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=56 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:54.252561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:54.253950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:54.308842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e4e37-842a-4b98-ac31-36fb3c21d156 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:54.312390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:54.313000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:54.313277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:54.313549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:54.315346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:54.376901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:54.378348 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:54.387747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1254/2498] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:40:57.475447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:57.476912 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:57.491649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387925e-14c1-4af2-afd9-fa76f0f8fc78 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:57.496267 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:57.496924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:57.497205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:57.497480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] PATH_INFO: `/v3/users/5378c85cd31c42dfbc39ce738a8b6dde` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:57.512681 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5378c85cd31c42dfbc39ce738a8b6dde)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:57.514597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:57.561526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5378c85cd31c42dfbc39ce738a8b6dde'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.563132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5378c85cd31c42dfbc39ce738a8b6dde'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.563765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5378c85cd31c42dfbc39ce738a8b6dde'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.576426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5378c85cd31c42dfbc39ce738a8b6dde'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.583137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5378c85cd31c42dfbc39ce738a8b6dde'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.595442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1245/2499] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:40:57 2022] DELETE /identity/v3/users/5378c85cd31c42dfbc39ce738a8b6dde => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:57.762502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:57.764596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:57.771459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:57.773064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:57.777423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51145d3-04f6-42fc-a0e0-109ebdf45933 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:57.830810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b3405-d8e2-4e45-9af6-dc0d778fa910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:57.831456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3405-d8e2-4e45-9af6-dc0d778fa910 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:57.831734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3405-d8e2-4e45-9af6-dc0d778fa910 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:57.832002 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3405-d8e2-4e45-9af6-dc0d778fa910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:57.833803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-259b3405-d8e2-4e45-9af6-dc0d778fa910 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:57.847398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb628a-5e2d-43d2-af22-ec883211e975 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:57.852067 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:57.852734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:57.853090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:57.853699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] PATH_INFO: `/v3/projects/96a7bb12d3ed4d1db2a58a03472483c2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:57.866936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a7bb12d3ed4d1db2a58a03472483c2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:57.870336 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:57.909236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:57.918110 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a7bb12d3ed4d1db2a58a03472483c2', 'request_id': 'req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.925579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a7bb12d3ed4d1db2a58a03472483c2', 'request_id': 'req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.926973 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:57.927709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b3405-d8e2-4e45-9af6-dc0d778fa910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:57.929042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b3405-d8e2-4e45-9af6-dc0d778fa910 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:57.932728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a7bb12d3ed4d1db2a58a03472483c2', 'request_id': 'req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.946778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] The token cache is being invalidate because project 96a7bb12d3ed4d1db2a58a03472483c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:40:57.947093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.947756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fa551f0-fd9b-4487-99b3-4d7db0ad2c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:57.949436 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1246/2500] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:40:57 2022] DELETE /identity/v3/projects/96a7bb12d3ed4d1db2a58a03472483c2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:40:57.952820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1255/2501] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:40:57.954256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a836-8eb4-499b-be65-8e51254f88ef None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:57.954662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a836-8eb4-499b-be65-8e51254f88ef None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:57.954891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a836-8eb4-499b-be65-8e51254f88ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:57.975009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a413a836-8eb4-499b-be65-8e51254f88ef None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:57.976129 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a413a836-8eb4-499b-be65-8e51254f88ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:58.047781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1247/2502] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:40:58.055621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73e571-f095-4b10-b9ff-fd12d21f29fd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.055912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73e571-f095-4b10-b9ff-fd12d21f29fd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.056192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73e571-f095-4b10-b9ff-fd12d21f29fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.075728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1f73e571-f095-4b10-b9ff-fd12d21f29fd None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:58.076930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1f73e571-f095-4b10-b9ff-fd12d21f29fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:58.084839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1256/2503] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:40:58.089036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413a836-8eb4-499b-be65-8e51254f88ef None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.102310 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413a836-8eb4-499b-be65-8e51254f88ef None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.106903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.107514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.107792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.108066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.108749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:40:58.109650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.111251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.126531 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1248/2504] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:40:58.127772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73e571-f095-4b10-b9ff-fd12d21f29fd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.136770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73e571-f095-4b10-b9ff-fd12d21f29fd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.141472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.142100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.142375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.142644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.143957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.145839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.178518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1257/2505] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:40:58 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:40:58.183386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.194923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678741a-f7f7-4cde-8643-00875aeb8d78 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.278108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.279851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.279851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.279851 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.280143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.283068 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.309368 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1249/2506] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:40:58 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:40:58.310856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.322079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167993e-7856-4268-9ec8-2efb4363f288 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.326835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.327468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.328272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.328664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.329434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.331321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.341898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1258/2507] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:58.346599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.357201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238ce42-392a-42f3-8cb6-d0ae3970bfb8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.362297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4a48-a9e5-42ad-8b80-5c645d0f1619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.362963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4a48-a9e5-42ad-8b80-5c645d0f1619 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.363245 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4a48-a9e5-42ad-8b80-5c645d0f1619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.363578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4a48-a9e5-42ad-8b80-5c645d0f1619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.364439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef4a48-a9e5-42ad-8b80-5c645d0f1619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.366202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef4a48-a9e5-42ad-8b80-5c645d0f1619 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.380918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1250/2508] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:58.382011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.393059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719be410-dc8e-44b2-9b28-bdc38d384b62 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.397696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.398355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.398638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.398952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] PATH_INFO: `/v3/projects/2e74e1e9130b42429ed6f5c8c7c288ee/users/17c57b905964464cbec9d68de63e5a8e/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.414214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e74e1e9130b42429ed6f5c8c7c288ee, user_id=17c57b905964464cbec9d68de63e5a8e, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.416873 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.430533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1259/2509] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:40:58 2022] PUT /identity/v3/projects/2e74e1e9130b42429ed6f5c8c7c288ee/users/17c57b905964464cbec9d68de63e5a8e/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:58.440761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d2f26-46d3-4462-994b-c46c79c639c3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.441048 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d2f26-46d3-4462-994b-c46c79c639c3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.441317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d2f26-46d3-4462-994b-c46c79c639c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.475264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-aa1d2f26-46d3-4462-994b-c46c79c639c3 None None] MFA Rules not processed for user `17c57b905964464cbec9d68de63e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:58.476464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aa1d2f26-46d3-4462-994b-c46c79c639c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:58.512501 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:58.545486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1251/2510] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:40:58.550095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.611658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f27c3-4736-4142-8dd7-2b591a3326b2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.616346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.616957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.617234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.617507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.618166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.620083 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.646292 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1260/2511] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:40:58 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:40:58.650422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d2f26-46d3-4462-994b-c46c79c639c3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.660935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d2f26-46d3-4462-994b-c46c79c639c3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.717115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.717724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.718001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.718279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.718988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.720902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.741594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1252/2512] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:40:58 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:40:58.745557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.756176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36262152-4b9b-4f7e-9450-1b2e0f1be9b5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.760755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.761362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.761642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.761913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.762646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.764228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.775096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1261/2513] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:58.779178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.792103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3307b-8806-48f2-91e4-4b4ca8011d18 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.796982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.797652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.797934 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.798339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] PATH_INFO: `/v3/projects/0d9c5ec9a3134b42b7c6966c48778a9b/users/050ed8006ae344a98103d9c9bcbd5bdd/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.814076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d9c5ec9a3134b42b7c6966c48778a9b, user_id=050ed8006ae344a98103d9c9bcbd5bdd, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.816755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.829597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1253/2514] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:40:58 2022] PUT /identity/v3/projects/0d9c5ec9a3134b42b7c6966c48778a9b/users/050ed8006ae344a98103d9c9bcbd5bdd/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:40:58.833601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.845808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd36f-0e8e-4a8e-917c-1c4b28568754 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.892768 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:58.905943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea62e6-7c0b-4782-a0c7-aea32f77f1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.906555 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea62e6-7c0b-4782-a0c7-aea32f77f1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.906832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea62e6-7c0b-4782-a0c7-aea32f77f1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.907126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea62e6-7c0b-4782-a0c7-aea32f77f1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.907803 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea62e6-7c0b-4782-a0c7-aea32f77f1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.909380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea62e6-7c0b-4782-a0c7-aea32f77f1cd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.918877 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1262/2515] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:40:58.923108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:58.933552 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad2545-9916-4012-987d-f3236e52e070 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:58.938661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:58.939357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.939632 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.939925 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] PATH_INFO: `/v3/projects/0d9c5ec9a3134b42b7c6966c48778a9b/users/050ed8006ae344a98103d9c9bcbd5bdd/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.941837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d9c5ec9a3134b42b7c6966c48778a9b, user_id=050ed8006ae344a98103d9c9bcbd5bdd, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:58.944488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:58.955173 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1254/2516] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:40:58 2022] PUT /identity/v3/projects/0d9c5ec9a3134b42b7c6966c48778a9b/users/050ed8006ae344a98103d9c9bcbd5bdd/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:40:58.966382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56070ea-64fc-4440-b701-431dc2e19a62 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:58.966664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56070ea-64fc-4440-b701-431dc2e19a62 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:58.966971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56070ea-64fc-4440-b701-431dc2e19a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:58.993627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b56070ea-64fc-4440-b701-431dc2e19a62 None None] MFA Rules not processed for user `050ed8006ae344a98103d9c9bcbd5bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:58.994811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b56070ea-64fc-4440-b701-431dc2e19a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:59.061441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1263/2517] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:40:59.073227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:59.075015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:59.136230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fd4fa-ee15-42da-bba5-d4d90790fe0c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:59.140835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e378c-8aec-49e9-9bfa-c76b70bd5624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:59.140835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e378c-8aec-49e9-9bfa-c76b70bd5624 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:59.140835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e378c-8aec-49e9-9bfa-c76b70bd5624 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:59.141165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e378c-8aec-49e9-9bfa-c76b70bd5624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:59.150236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e378c-8aec-49e9-9bfa-c76b70bd5624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:59.151676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e378c-8aec-49e9-9bfa-c76b70bd5624 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:59.161338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1255/2518] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:40:59.558913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56070ea-64fc-4440-b701-431dc2e19a62 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:59.623177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56070ea-64fc-4440-b701-431dc2e19a62 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:59.627696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:59.628334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:59.628615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:59.628890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] PATH_INFO: `/v3/users/17c57b905964464cbec9d68de63e5a8e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:59.630054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c57b905964464cbec9d68de63e5a8e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:59.632099 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:59.677529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c57b905964464cbec9d68de63e5a8e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:59.684532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c57b905964464cbec9d68de63e5a8e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:59.685188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c57b905964464cbec9d68de63e5a8e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:59.685793 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c57b905964464cbec9d68de63e5a8e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:59.692766 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c57b905964464cbec9d68de63e5a8e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:40:59.698035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:59.711042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1264/2519] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:40:59 2022] DELETE /identity/v3/users/17c57b905964464cbec9d68de63e5a8e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:40:59.715477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:59.715979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:59.716093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:59.735140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:40:59.736316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:59.764376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:40:59.795352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1256/2520] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:40:59.805970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:40:59.807408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:40:59.863864 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de43014-45d9-467e-806c-24704acc92a5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:40:59.867476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:40:59.868071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:40:59.868349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:40:59.868618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:40:59.878595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:40:59.880311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:40:59.891623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1265/2521] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:00.175285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:00.176757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:00.192296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97a66f-1632-4ce7-ae2b-a5d9b86f79ad None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:00.197210 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:00.197860 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:00.198138 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:00.198542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] PATH_INFO: `/v3/projects/2e74e1e9130b42429ed6f5c8c7c288ee` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:00.201195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e74e1e9130b42429ed6f5c8c7c288ee)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:00.204371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:00.254077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e74e1e9130b42429ed6f5c8c7c288ee', 'request_id': 'req-91430c2b-7362-4bba-ad8d-ba9f66ca5801'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.261311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e74e1e9130b42429ed6f5c8c7c288ee', 'request_id': 'req-91430c2b-7362-4bba-ad8d-ba9f66ca5801'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.267938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e74e1e9130b42429ed6f5c8c7c288ee', 'request_id': 'req-91430c2b-7362-4bba-ad8d-ba9f66ca5801'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.279980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] The token cache is being invalidate because project 2e74e1e9130b42429ed6f5c8c7c288ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:00.280270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.280951 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.282472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1257/2522] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:00 2022] DELETE /identity/v3/projects/2e74e1e9130b42429ed6f5c8c7c288ee => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:00.286631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:00.296140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d4a7b-5fb9-4eb6-b4ef-ed04b62695f8 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:00.354028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:00.354604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:00.354909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:00.355187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] PATH_INFO: `/v3/users/050ed8006ae344a98103d9c9bcbd5bdd` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:00.356351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050ed8006ae344a98103d9c9bcbd5bdd)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:00.358272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:00.406043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050ed8006ae344a98103d9c9bcbd5bdd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.412848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050ed8006ae344a98103d9c9bcbd5bdd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.413522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050ed8006ae344a98103d9c9bcbd5bdd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.414166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050ed8006ae344a98103d9c9bcbd5bdd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.421617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45bbdbc-c1d4-4d70-bee7-2fd1657dd511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050ed8006ae344a98103d9c9bcbd5bdd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.444487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1266/2523] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:00 2022] DELETE /identity/v3/users/050ed8006ae344a98103d9c9bcbd5bdd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:00.762757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:00.764223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:00.787484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91430c2b-7362-4bba-ad8d-ba9f66ca5801 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:00.877241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:00.877816 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:00.878094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:00.878366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] PATH_INFO: `/v3/projects/0d9c5ec9a3134b42b7c6966c48778a9b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:00.879537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d9c5ec9a3134b42b7c6966c48778a9b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:00.881420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:00.931174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d9c5ec9a3134b42b7c6966c48778a9b', 'request_id': 'req-2728da8a-a1f2-4315-8d9c-59a09b808e96'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.937635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d9c5ec9a3134b42b7c6966c48778a9b', 'request_id': 'req-2728da8a-a1f2-4315-8d9c-59a09b808e96'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.942170 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:00.946445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d9c5ec9a3134b42b7c6966c48778a9b', 'request_id': 'req-2728da8a-a1f2-4315-8d9c-59a09b808e96'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.962435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] The token cache is being invalidate because project 0d9c5ec9a3134b42b7c6966c48778a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:00.962745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.963444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2728da8a-a1f2-4315-8d9c-59a09b808e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:00.964999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1258/2524] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:00 2022] DELETE /identity/v3/projects/0d9c5ec9a3134b42b7c6966c48778a9b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:00.970673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa33d3-d0e3-45ec-8b0e-ce396308ce2b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:00.971193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa33d3-d0e3-45ec-8b0e-ce396308ce2b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:00.971303 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa33d3-d0e3-45ec-8b0e-ce396308ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:00.991005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-16fa33d3-d0e3-45ec-8b0e-ce396308ce2b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:00.992202 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-16fa33d3-d0e3-45ec-8b0e-ce396308ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:00.996528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:01.069188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1267/2525] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:01.077406 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d596de1-0947-4a61-95f6-d25d4c6f74d0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.077724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d596de1-0947-4a61-95f6-d25d4c6f74d0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.077988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d596de1-0947-4a61-95f6-d25d4c6f74d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.099758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2d596de1-0947-4a61-95f6-d25d4c6f74d0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:01.100978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2d596de1-0947-4a61-95f6-d25d4c6f74d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:01.112773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1259/2526] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:01.113774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa33d3-d0e3-45ec-8b0e-ce396308ce2b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.123737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa33d3-d0e3-45ec-8b0e-ce396308ce2b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.128374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.128934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.129229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.129458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.130373 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:01.130895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.132708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.146040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1268/2527] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:01.150328 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d596de1-0947-4a61-95f6-d25d4c6f74d0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.160349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d596de1-0947-4a61-95f6-d25d4c6f74d0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.165688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.166510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.166937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.167221 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.167920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.169828 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.203481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1260/2528] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:01 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:01.205183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.215307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7061f0b-0120-4d07-9a5f-472fa4cf665d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.279195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.279836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.280122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.281459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.282456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.285086 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.318113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1269/2529] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:01 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:01.321842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.341658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c1af-ce04-4722-9ac5-5c38168329c7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.347688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.348321 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.348603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.348921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.349567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.351416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.366665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1261/2530] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:01.369041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.382649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa1b98-20bc-46fe-80ab-98667ee23110 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.392172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936e023-ba41-4eb2-a26f-1d225147fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.393371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936e023-ba41-4eb2-a26f-1d225147fcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.393658 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936e023-ba41-4eb2-a26f-1d225147fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.393926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936e023-ba41-4eb2-a26f-1d225147fcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.394619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936e023-ba41-4eb2-a26f-1d225147fcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.396274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936e023-ba41-4eb2-a26f-1d225147fcb5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.411846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1270/2531] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:01.413250 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.426547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603787a8-1d1e-4963-ac63-1828574f1390 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.436656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.437468 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.437754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.438027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] PATH_INFO: `/v3/projects/0f9f506c35ab429b9dd3e45f5fba8cc8/users/f7c978931fdf41f1a157c27a04fee736/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.463440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9f506c35ab429b9dd3e45f5fba8cc8, user_id=f7c978931fdf41f1a157c27a04fee736, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.466181 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.481903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1262/2532] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:01 2022] PUT /identity/v3/projects/0f9f506c35ab429b9dd3e45f5fba8cc8/users/f7c978931fdf41f1a157c27a04fee736/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:01.492357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede1cfe-c594-4fe7-addd-1f7833d213ac None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.492646 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede1cfe-c594-4fe7-addd-1f7833d213ac None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.493019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede1cfe-c594-4fe7-addd-1f7833d213ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.536798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bede1cfe-c594-4fe7-addd-1f7833d213ac None None] MFA Rules not processed for user `f7c978931fdf41f1a157c27a04fee736`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:01.538023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bede1cfe-c594-4fe7-addd-1f7833d213ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:01.620648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1271/2533] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:01.625764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.688096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e4ea4-7ae6-42cf-98f1-7474736e382b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.692885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.693578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.693932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.694209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.694884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.696773 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.724279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1263/2534] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:01 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:01.728362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede1cfe-c594-4fe7-addd-1f7833d213ac None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.735104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:01.740545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede1cfe-c594-4fe7-addd-1f7833d213ac None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.808968 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.809601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.809880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.810153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.810828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.813355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.840971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1272/2535] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:01 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:01.844946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.860798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13b918-87a9-4bf2-8d26-44dd472586a2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.866594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.867271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.867563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.867859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.868596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.870291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.893779 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1264/2536] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:01.897954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.911702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6c05-4d10-41a5-addc-fb62ee720bed admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:01.917668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:01.918504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:01.918976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:01.919368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] PATH_INFO: `/v3/projects/a0f95ebe11d649fd9f50963793f6774f/users/3684b6e97a00459a966779d7938d38fb/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:01.946186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f95ebe11d649fd9f50963793f6774f, user_id=3684b6e97a00459a966779d7938d38fb, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:01.950528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:01.970191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1273/2537] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:01 2022] PUT /identity/v3/projects/a0f95ebe11d649fd9f50963793f6774f/users/3684b6e97a00459a966779d7938d38fb/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:01.974380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:01.989102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff66fc-8e83-4bff-bbec-80b96980909b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:02.094734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:02.104340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d1d82-9031-48c1-9f03-7b2ccd4b9f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:02.105499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1d82-9031-48c1-9f03-7b2ccd4b9f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:02.106007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1d82-9031-48c1-9f03-7b2ccd4b9f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:02.106676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1d82-9031-48c1-9f03-7b2ccd4b9f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:02.107752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d1d82-9031-48c1-9f03-7b2ccd4b9f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:02.110077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d1d82-9031-48c1-9f03-7b2ccd4b9f58 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:02.142908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1265/2538] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:02.145977 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:02.160103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7ecba-0ee2-4961-90e5-1454263cb8e0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:02.165525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:02.166242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:02.166579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:02.167017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] PATH_INFO: `/v3/projects/a0f95ebe11d649fd9f50963793f6774f/users/3684b6e97a00459a966779d7938d38fb/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:02.168854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f95ebe11d649fd9f50963793f6774f, user_id=3684b6e97a00459a966779d7938d38fb, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:02.172117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:02.194942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1274/2539] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:02 2022] PUT /identity/v3/projects/a0f95ebe11d649fd9f50963793f6774f/users/3684b6e97a00459a966779d7938d38fb/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:02.203101 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306f3f-bf38-4996-baa5-7b40fe05dc4a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:02.203529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306f3f-bf38-4996-baa5-7b40fe05dc4a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:02.203801 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306f3f-bf38-4996-baa5-7b40fe05dc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:02.256604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d7306f3f-bf38-4996-baa5-7b40fe05dc4a None None] MFA Rules not processed for user `3684b6e97a00459a966779d7938d38fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:02.257970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d7306f3f-bf38-4996-baa5-7b40fe05dc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:02.380624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1266/2540] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:02.392312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:02.394379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:02.464591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328eac-d79d-41c6-84b1-7afc03bb61e0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:02.468640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94402a-c59b-4d56-bab8-5d5187e957c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:02.469267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94402a-c59b-4d56-bab8-5d5187e957c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:02.469543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94402a-c59b-4d56-bab8-5d5187e957c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:02.469811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94402a-c59b-4d56-bab8-5d5187e957c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:02.481054 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94402a-c59b-4d56-bab8-5d5187e957c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:02.482593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94402a-c59b-4d56-bab8-5d5187e957c4 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:02.493808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1275/2541] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:03.936407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7306f3f-bf38-4996-baa5-7b40fe05dc4a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:04.003558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7306f3f-bf38-4996-baa5-7b40fe05dc4a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:04.008371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:04.009093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:04.009378 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:04.009652 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] PATH_INFO: `/v3/users/f7c978931fdf41f1a157c27a04fee736` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:04.011157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c978931fdf41f1a157c27a04fee736)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:04.013209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:04.079802 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c978931fdf41f1a157c27a04fee736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.080503 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c978931fdf41f1a157c27a04fee736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.081190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c978931fdf41f1a157c27a04fee736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.095706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c978931fdf41f1a157c27a04fee736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.102533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c978931fdf41f1a157c27a04fee736'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.111665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:04.118961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1267/2542] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:03 2022] DELETE /identity/v3/users/f7c978931fdf41f1a157c27a04fee736 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:04.123614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:04.124084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:04.124343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:04.144757 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:04.145973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:04.198551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:04.208286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1276/2543] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:04.217953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:04.219445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:04.280498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cfd29-f6c7-44c6-bbff-1a977f41a09a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:04.285282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:04.286677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:04.287102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:04.287499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:04.296480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:04.298472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:04.307861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1268/2544] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:04.578095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:04.579742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:04.591806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6d0cf-fd3b-42c8-8b5d-c8738452d797 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:04.596475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:04.597143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:04.597404 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:04.597794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] PATH_INFO: `/v3/projects/0f9f506c35ab429b9dd3e45f5fba8cc8` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:04.599049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9f506c35ab429b9dd3e45f5fba8cc8)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:04.601000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:04.648525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9f506c35ab429b9dd3e45f5fba8cc8', 'request_id': 'req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.654728 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9f506c35ab429b9dd3e45f5fba8cc8', 'request_id': 'req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.662810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9f506c35ab429b9dd3e45f5fba8cc8', 'request_id': 'req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.675072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] The token cache is being invalidate because project 0f9f506c35ab429b9dd3e45f5fba8cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:04.675586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.676552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.678064 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1277/2545] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:04 2022] DELETE /identity/v3/projects/0f9f506c35ab429b9dd3e45f5fba8cc8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:04.682243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:04.692422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c32b5-e8d5-4c55-a2de-2ecc6d16a97f service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:04.750698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:04.751373 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:04.751654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:04.751942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] PATH_INFO: `/v3/users/3684b6e97a00459a966779d7938d38fb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:04.753162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3684b6e97a00459a966779d7938d38fb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:04.755046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:04.805731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3684b6e97a00459a966779d7938d38fb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.806472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3684b6e97a00459a966779d7938d38fb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.807162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3684b6e97a00459a966779d7938d38fb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.823591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3684b6e97a00459a966779d7938d38fb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.831320 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3684b6e97a00459a966779d7938d38fb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:04.846118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1269/2546] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:04 2022] DELETE /identity/v3/users/3684b6e97a00459a966779d7938d38fb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:05.138933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.141265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:05.156455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b2aab-2aaf-4b79-9fe3-0ff97ddbba42 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.217573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.218222 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.218506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.219596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] PATH_INFO: `/v3/projects/a0f95ebe11d649fd9f50963793f6774f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.221013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f95ebe11d649fd9f50963793f6774f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.222982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.274759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f95ebe11d649fd9f50963793f6774f', 'request_id': 'req-9fa415a8-f42d-45de-bef4-0b592cd0aebd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:05.283784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f95ebe11d649fd9f50963793f6774f', 'request_id': 'req-9fa415a8-f42d-45de-bef4-0b592cd0aebd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:05.301494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.302499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f95ebe11d649fd9f50963793f6774f', 'request_id': 'req-9fa415a8-f42d-45de-bef4-0b592cd0aebd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:05.303153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:05.315271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] The token cache is being invalidate because project a0f95ebe11d649fd9f50963793f6774f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:05.315560 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:05.316200 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fa415a8-f42d-45de-bef4-0b592cd0aebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:05.317694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1278/2547] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:05 2022] DELETE /identity/v3/projects/a0f95ebe11d649fd9f50963793f6774f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:05.322482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507cfa0-4eb4-452a-818f-7315799aad31 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.322795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507cfa0-4eb4-452a-818f-7315799aad31 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.323115 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507cfa0-4eb4-452a-818f-7315799aad31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.327385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:05.342557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9507cfa0-4eb4-452a-818f-7315799aad31 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:05.343789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9507cfa0-4eb4-452a-818f-7315799aad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:05.352490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:05.368797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1d14-06cc-4fd1-a175-e07c5eb8844e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.408765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1279/2548] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:05.416246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e368c-c55a-44d2-ab25-c88c682d25da None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.416529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e368c-c55a-44d2-ab25-c88c682d25da None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.416797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e368c-c55a-44d2-ab25-c88c682d25da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.419345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.419969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.420252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.420527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.422317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:05.434517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.435969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.436582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-614e368c-c55a-44d2-ab25-c88c682d25da None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:05.438239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-614e368c-c55a-44d2-ab25-c88c682d25da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:05.446196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1270/2549] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:05.446408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1280/2550] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:05.450356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.460114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca9075-75e7-454c-a52d-a06c6a430bf9 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.464877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.465583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.465874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.466143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.466848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:05.467187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.468863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.480318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1271/2551] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:41:05.484530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e368c-c55a-44d2-ab25-c88c682d25da None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.493856 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e368c-c55a-44d2-ab25-c88c682d25da None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.498405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.499036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.499311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.499578 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.500753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.502629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.530005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1281/2552] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:05 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:05.534189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.544217 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e2fd-6380-4e09-8d40-38569e109586 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.601881 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.602743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.603036 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.603294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.604160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.606407 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.628933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1272/2553] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:05 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:05.633199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.644043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802ce7-9582-47ce-bf81-80333ded74e8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.648615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.649219 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.649489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.649756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.650437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.652045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.662721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1282/2554] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:05.666942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.676955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269a5c7-84c3-4ba7-aa3c-4746ce2bd8ae admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.681612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c808ba3-2952-478e-bb25-46847ee842e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.682329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c808ba3-2952-478e-bb25-46847ee842e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.682698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c808ba3-2952-478e-bb25-46847ee842e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.683304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c808ba3-2952-478e-bb25-46847ee842e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.684056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c808ba3-2952-478e-bb25-46847ee842e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.685645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c808ba3-2952-478e-bb25-46847ee842e7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.697098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1273/2555] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:05.701285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.712570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c6942-d648-4b06-97bc-1536261286d0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.717445 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.718166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.718503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.718737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] PATH_INFO: `/v3/projects/78cd53aea77e404f8ad2e7cdcd613fc1/users/949ea8d2114d480ea0a017c929344433/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.734361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78cd53aea77e404f8ad2e7cdcd613fc1, user_id=949ea8d2114d480ea0a017c929344433, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.737503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.752975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1283/2556] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:05 2022] PUT /identity/v3/projects/78cd53aea77e404f8ad2e7cdcd613fc1/users/949ea8d2114d480ea0a017c929344433/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:05.764480 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8287a7-c3c6-4228-aab7-7919b66472c3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.764770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8287a7-c3c6-4228-aab7-7919b66472c3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.765046 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8287a7-c3c6-4228-aab7-7919b66472c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.793761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8b8287a7-c3c6-4228-aab7-7919b66472c3 None None] MFA Rules not processed for user `949ea8d2114d480ea0a017c929344433`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:05.794943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8b8287a7-c3c6-4228-aab7-7919b66472c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:05.829128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:05.872388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1274/2557] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:05.877321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.944326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453ce1-af3a-4dc1-b338-975ac3b33002 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:05.949250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:05.949664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:05.949943 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:05.950215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:05.950896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:05.952846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:05.980165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1284/2558] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:05 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:05.984237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8287a7-c3c6-4228-aab7-7919b66472c3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:05.995542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8287a7-c3c6-4228-aab7-7919b66472c3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.053875 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.054453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.054756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.055075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.055775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.057730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.091917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1275/2559] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:05 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:06.093063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:06.103761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87e090-e48f-496b-b73e-182382db0655 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.108554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.109307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.110157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.110427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.111146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.112738 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.123323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1285/2560] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:06.128123 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:06.138354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b18ed-2e2a-4bc5-a0ad-b704f236bea3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.143617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.144281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.144561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.144834 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] PATH_INFO: `/v3/projects/8716cbe40b2c4a958c64fc5b32fa387d/users/156bfcee8e174315a179d088ddaae586/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.168817 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8716cbe40b2c4a958c64fc5b32fa387d, user_id=156bfcee8e174315a179d088ddaae586, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.171581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.185144 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1276/2561] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:06 2022] PUT /identity/v3/projects/8716cbe40b2c4a958c64fc5b32fa387d/users/156bfcee8e174315a179d088ddaae586/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:06.189209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:06.199864 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be590a4a-42ea-4e1d-b600-cfbca2cdb575 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.220829 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:06.260479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0134ad-a64f-4150-a665-5a33dce05891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.261132 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0134ad-a64f-4150-a665-5a33dce05891 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.261413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0134ad-a64f-4150-a665-5a33dce05891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.262177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0134ad-a64f-4150-a665-5a33dce05891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.262953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0134ad-a64f-4150-a665-5a33dce05891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.264498 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0134ad-a64f-4150-a665-5a33dce05891 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.275079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1286/2562] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:06.279440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:06.290738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7720981-8bfd-4c07-a5ec-88cd304614a7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.295491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.296167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.296445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.296718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] PATH_INFO: `/v3/projects/8716cbe40b2c4a958c64fc5b32fa387d/users/156bfcee8e174315a179d088ddaae586/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.298764 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8716cbe40b2c4a958c64fc5b32fa387d, user_id=156bfcee8e174315a179d088ddaae586, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.301411 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.312287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1277/2563] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:06 2022] PUT /identity/v3/projects/8716cbe40b2c4a958c64fc5b32fa387d/users/156bfcee8e174315a179d088ddaae586/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:06.323431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc0cb-9f88-4088-ab95-0c586e4385d4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.323830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc0cb-9f88-4088-ab95-0c586e4385d4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.324185 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc0cb-9f88-4088-ab95-0c586e4385d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.354411 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f3cfc0cb-9f88-4088-ab95-0c586e4385d4 None None] MFA Rules not processed for user `156bfcee8e174315a179d088ddaae586`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:06.356365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f3cfc0cb-9f88-4088-ab95-0c586e4385d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:06.438632 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1287/2564] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:06.450294 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:06.451918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:06.520625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f771840-3b7f-4d02-8476-791c1012bf1f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.524322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9fd9e-d597-448f-92e9-9ff5b871e93d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.524945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9fd9e-d597-448f-92e9-9ff5b871e93d service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.525226 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9fd9e-d597-448f-92e9-9ff5b871e93d service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.525500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9fd9e-d597-448f-92e9-9ff5b871e93d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.534791 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9fd9e-d597-448f-92e9-9ff5b871e93d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.536237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9fd9e-d597-448f-92e9-9ff5b871e93d service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.545591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1278/2565] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:06.575522 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfc0cb-9f88-4088-ab95-0c586e4385d4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:06.638367 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfc0cb-9f88-4088-ab95-0c586e4385d4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.643000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.643644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.643942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.644198 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] PATH_INFO: `/v3/users/949ea8d2114d480ea0a017c929344433` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.645374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=949ea8d2114d480ea0a017c929344433)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.647336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.696080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949ea8d2114d480ea0a017c929344433'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:06.697299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:06.704351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949ea8d2114d480ea0a017c929344433'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:06.706846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949ea8d2114d480ea0a017c929344433'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:06.707499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949ea8d2114d480ea0a017c929344433'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:06.715218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949ea8d2114d480ea0a017c929344433'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:06.733316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1288/2566] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:06 2022] DELETE /identity/v3/users/949ea8d2114d480ea0a017c929344433 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:06.738316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.738609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.738914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.758107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:06.759392 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:06.784140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:06.822590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1279/2567] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:06.838813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:06.840534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:06.907971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a34ab-f50b-4541-b3e3-5572a81cc16b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:06.914472 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:06.915432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:06.915894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:06.916346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:06.928875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:06.930656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:06.943270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1289/2568] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:07.252315 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:07.253814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:07.273433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380f9e5-f1a1-4145-b402-83dbf5f461d6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:07.279425 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:07.280078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:07.280770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:07.280770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] PATH_INFO: `/v3/projects/78cd53aea77e404f8ad2e7cdcd613fc1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:07.282484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=78cd53aea77e404f8ad2e7cdcd613fc1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:07.284420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:07.337481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78cd53aea77e404f8ad2e7cdcd613fc1', 'request_id': 'req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.343878 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78cd53aea77e404f8ad2e7cdcd613fc1', 'request_id': 'req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.352193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78cd53aea77e404f8ad2e7cdcd613fc1', 'request_id': 'req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.368820 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] The token cache is being invalidate because project 78cd53aea77e404f8ad2e7cdcd613fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:07.369203 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.370223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.372453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1280/2569] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:07 2022] DELETE /identity/v3/projects/78cd53aea77e404f8ad2e7cdcd613fc1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:07.376264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:07.386596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afceabb-fd68-437a-80fc-35c0d628cc65 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:07.450328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:07.451450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:07.451450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:07.451675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] PATH_INFO: `/v3/users/156bfcee8e174315a179d088ddaae586` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:07.453140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=156bfcee8e174315a179d088ddaae586)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:07.454782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:07.518272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156bfcee8e174315a179d088ddaae586'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.526747 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156bfcee8e174315a179d088ddaae586'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.527530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156bfcee8e174315a179d088ddaae586'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.528140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156bfcee8e174315a179d088ddaae586'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.538249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ee7da0a1-6a52-4c5c-809d-c1cafba7b885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156bfcee8e174315a179d088ddaae586'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:07.544611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:07.565815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1290/2570] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:07 2022] DELETE /identity/v3/users/156bfcee8e174315a179d088ddaae586 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:07.868158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:07.869861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:07.890029 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd1be8-3775-49a6-9f4f-c9b2a53c3d5a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:07.982627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:07.983326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:07.983608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:07.983886 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] PATH_INFO: `/v3/projects/8716cbe40b2c4a958c64fc5b32fa387d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:07.985601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8716cbe40b2c4a958c64fc5b32fa387d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:07.987564 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.043819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8716cbe40b2c4a958c64fc5b32fa387d', 'request_id': 'req-5ec9262d-312a-421f-b568-70de29119f9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.050733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8716cbe40b2c4a958c64fc5b32fa387d', 'request_id': 'req-5ec9262d-312a-421f-b568-70de29119f9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.052304 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:08.059405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8716cbe40b2c4a958c64fc5b32fa387d', 'request_id': 'req-5ec9262d-312a-421f-b568-70de29119f9c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.075731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] The token cache is being invalidate because project 8716cbe40b2c4a958c64fc5b32fa387d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:08.076365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.077164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5ec9262d-312a-421f-b568-70de29119f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.079982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1281/2571] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:07 2022] DELETE /identity/v3/projects/8716cbe40b2c4a958c64fc5b32fa387d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:08.084969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c3eb-09da-47a9-8cd0-435663e9f328 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.085329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c3eb-09da-47a9-8cd0-435663e9f328 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.085601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c3eb-09da-47a9-8cd0-435663e9f328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.105673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-43d5c3eb-09da-47a9-8cd0-435663e9f328 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:08.106901 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-43d5c3eb-09da-47a9-8cd0-435663e9f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:08.203131 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1291/2572] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:08.211072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe424b-db6b-4401-bcc8-fd4788d24407 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.212081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe424b-db6b-4401-bcc8-fd4788d24407 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.212081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe424b-db6b-4401-bcc8-fd4788d24407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.241829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b7fe424b-db6b-4401-bcc8-fd4788d24407 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:08.243908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b7fe424b-db6b-4401-bcc8-fd4788d24407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:08.264387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1282/2573] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:08.268859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5c3eb-09da-47a9-8cd0-435663e9f328 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.288451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5c3eb-09da-47a9-8cd0-435663e9f328 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.299769 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.300889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.301381 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.301859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.302785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:08.303363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.305258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.323174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1292/2574] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:41:08.326809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe424b-db6b-4401-bcc8-fd4788d24407 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.348089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe424b-db6b-4401-bcc8-fd4788d24407 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.353035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.353657 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.354357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.354357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.355062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.357060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.389887 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1283/2575] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:08 2022] POST /identity/v3/projects => generated 409 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:08.394091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.404744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9f213-5599-4442-963d-71cb5be15996 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.475680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.476344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.476627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.476903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.477581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.479629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.501684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1293/2576] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:08 2022] POST /identity/v3/users => generated 305 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:08.506286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.516619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f0ee-c53a-4a65-81a9-a824785387e8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.521241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.521852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.522148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.522400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.523130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.524698 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.535337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1284/2577] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:08.539642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.552949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44692e06-ad81-4de0-8e1a-93199ae510db admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.557645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569f9d-13ec-45b1-b9ff-d56ae99d1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.558519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569f9d-13ec-45b1-b9ff-d56ae99d1c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.559024 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569f9d-13ec-45b1-b9ff-d56ae99d1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.559145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569f9d-13ec-45b1-b9ff-d56ae99d1c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.559784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569f9d-13ec-45b1-b9ff-d56ae99d1c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.561377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569f9d-13ec-45b1-b9ff-d56ae99d1c76 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.572976 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1294/2578] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:08.577859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.589606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad6b16-fea3-4530-bcba-7c9c38560de5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.595064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.595735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.596012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.596285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] PATH_INFO: `/v3/projects/4ad53e497b9940d7b95422bdbb8f9899/users/a4fdb24d6be14cfb9f9a673e48187c21/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.615532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad53e497b9940d7b95422bdbb8f9899, user_id=a4fdb24d6be14cfb9f9a673e48187c21, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.618235 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.631628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1285/2579] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:08 2022] PUT /identity/v3/projects/4ad53e497b9940d7b95422bdbb8f9899/users/a4fdb24d6be14cfb9f9a673e48187c21/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:08.644124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.644419 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.644744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.676218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] MFA Rules not processed for user `a4fdb24d6be14cfb9f9a673e48187c21`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:08.677663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:08.740188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:08.767355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1295/2580] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:08 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:08.772350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.852193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.853716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:08.861492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fdd74-0c2b-46ce-bbc2-a4829671e7d4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.867455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.868214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.868582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.869043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.869674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.870413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923fd5e-d300-43c1-a1ba-5314d0390e42 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.871620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.875319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:08.876046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:08.876398 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:08.876737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] PATH_INFO: `/v3/users/be34f77da4d8407e9a4333bc71f64595` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:08.878022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be34f77da4d8407e9a4333bc71f64595)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:08.880048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:08.903765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1286/2581] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:08 2022] POST /identity/v3/projects => generated 407 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:08.908710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:08.931091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be34f77da4d8407e9a4333bc71f64595'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.938413 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be34f77da4d8407e9a4333bc71f64595'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.939159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be34f77da4d8407e9a4333bc71f64595'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.939789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be34f77da4d8407e9a4333bc71f64595'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.948307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be34f77da4d8407e9a4333bc71f64595'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:08.966668 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1296/2582] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:08 2022] DELETE /identity/v3/users/be34f77da4d8407e9a4333bc71f64595 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:08.975472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0af7f-bbde-4140-baf6-e8553c31bec5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:08.983491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:09.035542 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.036118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.036404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.036683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.037382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.039385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.060872 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1287/2583] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:08 2022] POST /identity/v3/users => generated 310 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:09.065046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.066478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.080355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef28c6-b4c4-4c0a-ba5f-f0dce570a819 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.085346 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.085945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.086229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.086508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.087229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.088827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.099524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1297/2584] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:09.103781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.113742 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93da48-6f2e-4fe2-a475-fac242a10fe2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.118638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.119347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.119627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.119898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] PATH_INFO: `/v3/projects/ee2c0c416935435d93fe2a46f528a2fa/users/bcd85f1f0e594404bfeac3b1cfcf5eb1/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.137935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee2c0c416935435d93fe2a46f528a2fa, user_id=bcd85f1f0e594404bfeac3b1cfcf5eb1, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.141330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.152550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.154396 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.157504 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1288/2585] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:09 2022] PUT /identity/v3/projects/ee2c0c416935435d93fe2a46f528a2fa/users/bcd85f1f0e594404bfeac3b1cfcf5eb1/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:09.161447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.170483 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba0660-e256-44ca-a557-a9651597cf8a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.236296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.236709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d8129-8385-4ace-8579-0f703a204223 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.236950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.237228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.237503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] PATH_INFO: `/v3/projects/2f20dc7bdc3a4448a56a76f1efbec596` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.238660 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f20dc7bdc3a4448a56a76f1efbec596)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.240604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.241431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.242060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.242345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.242624 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.243972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.245586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.256083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1289/2586] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:09.260161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.270574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c3056-5fc7-4a37-8ce7-6dffac3953dd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.275662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b199e-ec14-419e-bef7-4d2b267bf51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.276526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b199e-ec14-419e-bef7-4d2b267bf51a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.276810 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b199e-ec14-419e-bef7-4d2b267bf51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.277088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b199e-ec14-419e-bef7-4d2b267bf51a admin admin] PATH_INFO: `/v3/projects/ee2c0c416935435d93fe2a46f528a2fa/users/bcd85f1f0e594404bfeac3b1cfcf5eb1/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.279086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b199e-ec14-419e-bef7-4d2b267bf51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee2c0c416935435d93fe2a46f528a2fa, user_id=bcd85f1f0e594404bfeac3b1cfcf5eb1, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.282218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b199e-ec14-419e-bef7-4d2b267bf51a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.286264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:09.289455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f20dc7bdc3a4448a56a76f1efbec596', 'request_id': 'req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:09.292959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1290/2587] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:09 2022] PUT /identity/v3/projects/ee2c0c416935435d93fe2a46f528a2fa/users/bcd85f1f0e594404bfeac3b1cfcf5eb1/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:09.300334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f20dc7bdc3a4448a56a76f1efbec596', 'request_id': 'req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:09.303983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.304420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.304641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.307678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f20dc7bdc3a4448a56a76f1efbec596', 'request_id': 'req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:09.318477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] The token cache is being invalidate because project 2f20dc7bdc3a4448a56a76f1efbec596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:09.318767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:09.319449 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-19ba00e0-3bec-43df-bfc4-1db8c99f70c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:09.321205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1298/2588] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:09 2022] DELETE /identity/v3/projects/2f20dc7bdc3a4448a56a76f1efbec596 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:09.325710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5631115-1a02-4bc2-b28f-b6e95f6444e9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.326081 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5631115-1a02-4bc2-b28f-b6e95f6444e9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.326433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5631115-1a02-4bc2-b28f-b6e95f6444e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.333390 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] MFA Rules not processed for user `bcd85f1f0e594404bfeac3b1cfcf5eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:09.335427 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.345994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f5631115-1a02-4bc2-b28f-b6e95f6444e9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:09.347181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f5631115-1a02-4bc2-b28f-b6e95f6444e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.382756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:09.405649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1299/2589] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:09.406686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1291/2590] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:09.413960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79508a32-f66b-42ff-8c92-ac64efe9ff82 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.414254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79508a32-f66b-42ff-8c92-ac64efe9ff82 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.414528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79508a32-f66b-42ff-8c92-ac64efe9ff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.423394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.424788 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.434456 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-79508a32-f66b-42ff-8c92-ac64efe9ff82 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:09.435680 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-79508a32-f66b-42ff-8c92-ac64efe9ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.444158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1300/2591] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:09.448361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79508a32-f66b-42ff-8c92-ac64efe9ff82 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.458290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79508a32-f66b-42ff-8c92-ac64efe9ff82 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.463153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.463839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.464126 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.464430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.465127 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:09.465433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.467255 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.480159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1301/2592] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:41:09.485052 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.486540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e3a7a-086a-44fc-b1f9-f8f4e000c0ae None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.490258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.490893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.491673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.491673 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.494833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8334b-710a-43ff-bf7e-bbf8ef2b0169 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.499402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.500032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.500311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.500565 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.500693 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.501208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.501984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.503085 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.511405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1292/2593] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:09 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:41:09.534125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1302/2594] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:09.535269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.545395 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10cc4-8212-45e0-8ea5-f2bd5cbf977f service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.606207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.606945 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.607229 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.607522 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.608243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.610326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.632405 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1293/2595] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:09.636913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.647972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513182f-0dab-4817-9254-8407ee65c213 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.652686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.653323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.653604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.653876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.654561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.656950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.666790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1303/2596] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:09.671098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.682133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edf11d-e37c-4abf-b584-6af9765811ca admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.682322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.688345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.688982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.689264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.689540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.690252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.691895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.702508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1294/2597] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:09.706908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.717685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d8edc-9780-4ea8-9587-7a4101661858 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.723317 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6fcfa-ec65-4e4c-8b04-4c891cd9aee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.724010 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fcfa-ec65-4e4c-8b04-4c891cd9aee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.724296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fcfa-ec65-4e4c-8b04-4c891cd9aee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.724576 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6fcfa-ec65-4e4c-8b04-4c891cd9aee1 admin admin] PATH_INFO: `/v3/projects/0b643966e7d84ccd90e7e413aee3b3c5/users/d52bf42afce847b5a9992fbc87eead8a/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.741923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5baea-a514-45e3-a37c-7423c4e69f8f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.742266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6fcfa-ec65-4e4c-8b04-4c891cd9aee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b643966e7d84ccd90e7e413aee3b3c5, user_id=d52bf42afce847b5a9992fbc87eead8a, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.744939 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6fcfa-ec65-4e4c-8b04-4c891cd9aee1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.745919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.746547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.746853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.747173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.749108 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.758915 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1295/2598] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:09 2022] PUT /identity/v3/projects/0b643966e7d84ccd90e7e413aee3b3c5/users/d52bf42afce847b5a9992fbc87eead8a/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:09.770093 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63971ec-dbe9-49ad-b2c2-08c59fab9081 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.770382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63971ec-dbe9-49ad-b2c2-08c59fab9081 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.770648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63971ec-dbe9-49ad-b2c2-08c59fab9081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.785232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:09.801335 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b63971ec-dbe9-49ad-b2c2-08c59fab9081 None None] MFA Rules not processed for user `d52bf42afce847b5a9992fbc87eead8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:09.802562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b63971ec-dbe9-49ad-b2c2-08c59fab9081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:09.814417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.815923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.824870 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1304/2599] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:09 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:41:09.833670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:09.876804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1296/2600] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:09.881447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:09.892354 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000da953-d5fa-4aee-8338-6c27032fd609 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:09.952506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:09.953220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:09.953538 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:09.953828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:09.954524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:09.956679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:09.985257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1305/2601] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:09.989384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63971ec-dbe9-49ad-b2c2-08c59fab9081 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:10.002941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63971ec-dbe9-49ad-b2c2-08c59fab9081 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:10.067282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:10.075370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:10.075706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:10.075960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:10.076784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:10.078933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:10.103055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1297/2602] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:09 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:10.106817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:10.117994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c523-666a-4a58-9195-156c7babf8f1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:10.122899 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57502621-2375-497e-b710-53a889194338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:10.123519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57502621-2375-497e-b710-53a889194338 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:10.123796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57502621-2375-497e-b710-53a889194338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:10.124062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57502621-2375-497e-b710-53a889194338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:10.124755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57502621-2375-497e-b710-53a889194338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:10.127743 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57502621-2375-497e-b710-53a889194338 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:10.139177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1306/2603] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:10.143761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:10.154688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911845a-ca5f-461e-9ff7-921cab4fc470 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:10.160428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:10.161185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:10.161666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:10.162054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] PATH_INFO: `/v3/projects/aae38528ef9d4e26afcfd1236b258a7a/users/c8f4571d49e147ddb1f47cd7f9027aec/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:10.179946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae38528ef9d4e26afcfd1236b258a7a, user_id=c8f4571d49e147ddb1f47cd7f9027aec, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:10.182852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:10.198432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1298/2604] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:10 2022] PUT /identity/v3/projects/aae38528ef9d4e26afcfd1236b258a7a/users/c8f4571d49e147ddb1f47cd7f9027aec/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:10.202458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57502621-2375-497e-b710-53a889194338 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:10.213579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57502621-2375-497e-b710-53a889194338 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:10.286986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d4d4-b297-4dc2-8fd9-0db26ab9bc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:10.287622 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d4d4-b297-4dc2-8fd9-0db26ab9bc80 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:10.287905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d4d4-b297-4dc2-8fd9-0db26ab9bc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:10.288183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d4d4-b297-4dc2-8fd9-0db26ab9bc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:10.288872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d4d4-b297-4dc2-8fd9-0db26ab9bc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:10.290488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d4d4-b297-4dc2-8fd9-0db26ab9bc80 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:10.303879 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1307/2605] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:10.304981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:10.315981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258519f-bf07-4eef-9305-cd51f414be4c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:10.320733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:10.321402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:10.321688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:10.321971 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] PATH_INFO: `/v3/projects/aae38528ef9d4e26afcfd1236b258a7a/users/c8f4571d49e147ddb1f47cd7f9027aec/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:10.324020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae38528ef9d4e26afcfd1236b258a7a, user_id=c8f4571d49e147ddb1f47cd7f9027aec, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:10.326621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:10.337641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1299/2606] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:10 2022] PUT /identity/v3/projects/aae38528ef9d4e26afcfd1236b258a7a/users/c8f4571d49e147ddb1f47cd7f9027aec/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:10.348798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be84e-9a4e-4002-96a2-8a1b2ad320e5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:10.349159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be84e-9a4e-4002-96a2-8a1b2ad320e5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:10.349527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be84e-9a4e-4002-96a2-8a1b2ad320e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:10.378717 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-716be84e-9a4e-4002-96a2-8a1b2ad320e5 None None] MFA Rules not processed for user `c8f4571d49e147ddb1f47cd7f9027aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:10.380029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-716be84e-9a4e-4002-96a2-8a1b2ad320e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:10.416415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:10.456068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1308/2607] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:10.467385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:10.469756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:10.528775 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de4201-ee2f-4b88-948d-36ae8418efb4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:10.532941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fdb16-5728-4689-9c61-895de9dec614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:10.533548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fdb16-5728-4689-9c61-895de9dec614 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:10.533823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fdb16-5728-4689-9c61-895de9dec614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:10.534091 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fdb16-5728-4689-9c61-895de9dec614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:10.543246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fdb16-5728-4689-9c61-895de9dec614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:10.544650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fdb16-5728-4689-9c61-895de9dec614 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:10.554222 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1300/2608] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:10.966458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be84e-9a4e-4002-96a2-8a1b2ad320e5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:11.027655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be84e-9a4e-4002-96a2-8a1b2ad320e5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:11.032048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:11.032679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:11.032959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:11.033232 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] PATH_INFO: `/v3/users/d52bf42afce847b5a9992fbc87eead8a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:11.034379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52bf42afce847b5a9992fbc87eead8a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:11.036337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:11.082862 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52bf42afce847b5a9992fbc87eead8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.089806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52bf42afce847b5a9992fbc87eead8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.090496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52bf42afce847b5a9992fbc87eead8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.091140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52bf42afce847b5a9992fbc87eead8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.098487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52bf42afce847b5a9992fbc87eead8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.115431 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1309/2609] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:10 2022] DELETE /identity/v3/users/d52bf42afce847b5a9992fbc87eead8a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:11.119943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:11.120346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:11.120579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:11.139375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:11.140410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:11.161663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:11.206896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1301/2610] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:11.215539 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:11.216950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:11.277475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffdcf6-32b1-4f32-b650-d06540becae8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:11.282006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:11.282593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:11.282888 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:11.283181 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:11.289854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:11.294719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:11.296478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:11.308237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1310/2611] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:11.576027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:11.577908 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:11.607404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536c2b-71fb-4d1b-9cce-a2877ca3c4ab None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:11.614906 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:11.615824 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:11.616164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:11.616550 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] PATH_INFO: `/v3/projects/0b643966e7d84ccd90e7e413aee3b3c5` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:11.617976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b643966e7d84ccd90e7e413aee3b3c5)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:11.620042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:11.672911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b643966e7d84ccd90e7e413aee3b3c5', 'request_id': 'req-85f9be15-3691-461e-9bb8-f26fc0505e71'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.679089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b643966e7d84ccd90e7e413aee3b3c5', 'request_id': 'req-85f9be15-3691-461e-9bb8-f26fc0505e71'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.684874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b643966e7d84ccd90e7e413aee3b3c5', 'request_id': 'req-85f9be15-3691-461e-9bb8-f26fc0505e71'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.697953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] The token cache is being invalidate because project 0b643966e7d84ccd90e7e413aee3b3c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:11.698243 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.698902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.700535 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1302/2612] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:11 2022] DELETE /identity/v3/projects/0b643966e7d84ccd90e7e413aee3b3c5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:11.705066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:11.715046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd17e5-e454-45fe-847a-8a8f06528c21 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:11.774982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:11.775629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:11.775910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:11.776182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] PATH_INFO: `/v3/users/c8f4571d49e147ddb1f47cd7f9027aec` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:11.777339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f4571d49e147ddb1f47cd7f9027aec)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:11.779274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:11.826574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f4571d49e147ddb1f47cd7f9027aec'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.833569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f4571d49e147ddb1f47cd7f9027aec'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.834249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f4571d49e147ddb1f47cd7f9027aec'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.834855 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f4571d49e147ddb1f47cd7f9027aec'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.846627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-767e52ea-f30c-45e7-982b-633d3b8ece69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f4571d49e147ddb1f47cd7f9027aec'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:11.870964 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1311/2613] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:11 2022] DELETE /identity/v3/users/c8f4571d49e147ddb1f47cd7f9027aec => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:12.148382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.149948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:12.163919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9be15-3691-461e-9bb8-f26fc0505e71 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.221244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.221892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.222177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.222455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] PATH_INFO: `/v3/projects/aae38528ef9d4e26afcfd1236b258a7a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.223741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae38528ef9d4e26afcfd1236b258a7a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.225660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.269891 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae38528ef9d4e26afcfd1236b258a7a', 'request_id': 'req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:12.271720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:12.276219 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae38528ef9d4e26afcfd1236b258a7a', 'request_id': 'req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:12.283678 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae38528ef9d4e26afcfd1236b258a7a', 'request_id': 'req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:12.295433 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] The token cache is being invalidate because project aae38528ef9d4e26afcfd1236b258a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:12.295719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:12.296361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-95c7c0ed-6b6c-44d9-8628-7977fc0ed6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:12.298447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1303/2614] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:12 2022] DELETE /identity/v3/projects/aae38528ef9d4e26afcfd1236b258a7a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:12.303242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca805c-e263-43f5-8080-f56ecf2ee516 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.303557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca805c-e263-43f5-8080-f56ecf2ee516 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.303832 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca805c-e263-43f5-8080-f56ecf2ee516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.322920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d5ca805c-e263-43f5-8080-f56ecf2ee516 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:12.324103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d5ca805c-e263-43f5-8080-f56ecf2ee516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:12.379993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1312/2615] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:12.387560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b79d2-5c72-4d7a-9b4d-8df54534dd43 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.387854 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b79d2-5c72-4d7a-9b4d-8df54534dd43 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.388133 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b79d2-5c72-4d7a-9b4d-8df54534dd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.408593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-102b79d2-5c72-4d7a-9b4d-8df54534dd43 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:12.409790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-102b79d2-5c72-4d7a-9b4d-8df54534dd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:12.418130 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1304/2616] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:12.422956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca805c-e263-43f5-8080-f56ecf2ee516 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.432591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca805c-e263-43f5-8080-f56ecf2ee516 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.437385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.437987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.438266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.438534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.439323 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:12.439627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.441205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.449527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:12.453201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1313/2617] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:12.458702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b79d2-5c72-4d7a-9b4d-8df54534dd43 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.467871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b79d2-5c72-4d7a-9b4d-8df54534dd43 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.473148 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.473748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.474026 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.474301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.475001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.476880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.504200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1305/2618] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:12 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:12.508269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.518606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384159c-b18f-49ac-90fb-bbfd9b34639f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.571822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.572423 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.572700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.572972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.573620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.575551 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.597428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1314/2619] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:12 2022] POST /identity/v3/users => generated 279 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:12.601512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.612464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca63aa-3e98-4166-bccb-dcf6ab0804af admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.617060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.617665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.617941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.618209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.618916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.620473 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.630212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1306/2620] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:12.634138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.644478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2a74-c7b2-48e4-9d45-cebede197be5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.649007 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3d571-395f-4962-889b-65dcce471878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.649598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3d571-395f-4962-889b-65dcce471878 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.649872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3d571-395f-4962-889b-65dcce471878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.650141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3d571-395f-4962-889b-65dcce471878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.650816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3d571-395f-4962-889b-65dcce471878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.652446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3d571-395f-4962-889b-65dcce471878 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.663344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1315/2621] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:12.667334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.677117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d8c05-dce5-40d0-98b5-7eac31d3515b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.682000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.682717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.683306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.683608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] PATH_INFO: `/v3/projects/a0a3d1d304fd432997f1577268a7e2ae/users/4db1aaa0d9014e95984b7fe85852e1cb/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.697909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a3d1d304fd432997f1577268a7e2ae, user_id=4db1aaa0d9014e95984b7fe85852e1cb, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.700573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.713586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1307/2622] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:12 2022] PUT /identity/v3/projects/a0a3d1d304fd432997f1577268a7e2ae/users/4db1aaa0d9014e95984b7fe85852e1cb/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:12.724491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e007de-7c1e-4397-9562-115311d98170 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.724961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e007de-7c1e-4397-9562-115311d98170 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.725060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e007de-7c1e-4397-9562-115311d98170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.752865 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d5e007de-7c1e-4397-9562-115311d98170 None None] MFA Rules not processed for user `4db1aaa0d9014e95984b7fe85852e1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:12.754059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d5e007de-7c1e-4397-9562-115311d98170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:12.817191 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1316/2623] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:12.821745 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.883684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eff981-ac1e-4b47-a6e8-7c18f76fd0ae admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.888213 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.888830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.889109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.889382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.890039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.891974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:12.918556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1308/2624] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:12 2022] POST /identity/v3/projects => generated 353 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:12.923283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e007de-7c1e-4397-9562-115311d98170 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:12.934322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e007de-7c1e-4397-9562-115311d98170 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:12.992352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:12.992971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:12.993244 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:12.993514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:12.994171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:12.996078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:13.013173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:13.022613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1317/2625] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:12 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:13.023807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:13.034604 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac481a-549c-4096-912d-7c796ab33230 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:13.039285 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:13.039892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:13.040175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:13.040478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:13.041164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:13.042733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:13.053679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1309/2626] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:13.057619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:13.067608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa385e-4278-4d8a-af9e-3cf19f168d54 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:13.072169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:13.072813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:13.073097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:13.073371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] PATH_INFO: `/v3/projects/2af661efad174713bc793f00e60c2285/users/a0843b785b6145629bc45bc9cbc42873/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:13.087584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af661efad174713bc793f00e60c2285, user_id=a0843b785b6145629bc45bc9cbc42873, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:13.090212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:13.104050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1318/2627] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:13 2022] PUT /identity/v3/projects/2af661efad174713bc793f00e60c2285/users/a0843b785b6145629bc45bc9cbc42873/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:13.107874 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:13.118183 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb0b10-91e8-4a5c-b63e-7c2880bdb8d2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:13.162648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:13.174968 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a26d5-e5b6-4e7b-b99b-5ffba7540c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:13.175579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a26d5-e5b6-4e7b-b99b-5ffba7540c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:13.175861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a26d5-e5b6-4e7b-b99b-5ffba7540c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:13.176136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a26d5-e5b6-4e7b-b99b-5ffba7540c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:13.176929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a26d5-e5b6-4e7b-b99b-5ffba7540c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:13.178536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a26d5-e5b6-4e7b-b99b-5ffba7540c88 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:13.188338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1310/2628] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:13.192461 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:13.204249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed1126-ef74-400c-a828-824edb3fb493 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:13.209938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:13.210611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:13.210914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:13.211197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] PATH_INFO: `/v3/projects/2af661efad174713bc793f00e60c2285/users/a0843b785b6145629bc45bc9cbc42873/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:13.213275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af661efad174713bc793f00e60c2285, user_id=a0843b785b6145629bc45bc9cbc42873, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:13.217171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:13.228254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1319/2629] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:13 2022] PUT /identity/v3/projects/2af661efad174713bc793f00e60c2285/users/a0843b785b6145629bc45bc9cbc42873/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:13.239252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01b32d-76dd-4663-914b-602b115120b7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:13.239541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01b32d-76dd-4663-914b-602b115120b7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:13.239815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01b32d-76dd-4663-914b-602b115120b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:13.267023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3d01b32d-76dd-4663-914b-602b115120b7 None None] MFA Rules not processed for user `a0843b785b6145629bc45bc9cbc42873`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:13.268223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3d01b32d-76dd-4663-914b-602b115120b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:13.335713 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1311/2630] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:13.347448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:13.349152 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:13.404799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780b388-6842-4a01-9aa2-14a7e5a3eff1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:13.409016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:13.409623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:13.409903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:13.410174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:13.419212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:13.420586 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:13.430729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1320/2631] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:13.503950 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01b32d-76dd-4663-914b-602b115120b7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:13.513415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01b32d-76dd-4663-914b-602b115120b7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:13.517559 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904c49d-b1d2-4f23-8084-0ffbf618c406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:13.518164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904c49d-b1d2-4f23-8084-0ffbf618c406 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:13.518446 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904c49d-b1d2-4f23-8084-0ffbf618c406 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:13.518721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904c49d-b1d2-4f23-8084-0ffbf618c406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:13.580895 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904c49d-b1d2-4f23-8084-0ffbf618c406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:13.582329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904c49d-b1d2-4f23-8084-0ffbf618c406 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:13.591583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1312/2632] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:16.497605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:16.499049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:16.566860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fafa6-3fa8-45ae-96f8-109937e7436a service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:16.571486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:16.572124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:16.572402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:16.572682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] PATH_INFO: `/v3/users/726bbf45500c43db83300419f6390769` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:16.573839 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=726bbf45500c43db83300419f6390769)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:16.575782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:16.621716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726bbf45500c43db83300419f6390769'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:16.629377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726bbf45500c43db83300419f6390769'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:16.630062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726bbf45500c43db83300419f6390769'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:16.630715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726bbf45500c43db83300419f6390769'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:16.639206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726bbf45500c43db83300419f6390769'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:16.641822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:16.662210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1321/2633] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:16 2022] DELETE /identity/v3/users/726bbf45500c43db83300419f6390769 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:16.666671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:16.667174 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:16.667276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:16.687086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:16.688299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:16.748741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1313/2634] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:16.759595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:16.761010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:16.824287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00218-c649-4532-81b9-8097e258ceaa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:16.828001 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:16.828631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:16.828916 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:16.829190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:16.838315 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:16.839782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:16.852212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1322/2635] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:17.147167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:17.148609 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:17.162215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744e672-e8d2-40ac-ab8b-ffaf6ad0b359 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:17.168240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:17.169002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:17.169318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:17.169551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] PATH_INFO: `/v3/projects/3f51a9db8177444e8e69f8954692204e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:17.170770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f51a9db8177444e8e69f8954692204e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:17.173004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:17.182718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:17.232843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f51a9db8177444e8e69f8954692204e', 'request_id': 'req-7c333b71-5630-40fc-9e0c-dc97c03cc12b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.240776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f51a9db8177444e8e69f8954692204e', 'request_id': 'req-7c333b71-5630-40fc-9e0c-dc97c03cc12b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.246848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f51a9db8177444e8e69f8954692204e', 'request_id': 'req-7c333b71-5630-40fc-9e0c-dc97c03cc12b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.261400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] The token cache is being invalidate because project 3f51a9db8177444e8e69f8954692204e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:17.262256 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.262382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.263899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1314/2636] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:17 2022] DELETE /identity/v3/projects/3f51a9db8177444e8e69f8954692204e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:17.268379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:17.278422 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fd067-3f47-4846-9755-1bd4fe31888a service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:17.346459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:17.347158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:17.347437 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:17.347724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] PATH_INFO: `/v3/users/85e767005a2e4158b004bf85270b1689` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:17.348893 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e767005a2e4158b004bf85270b1689)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:17.350819 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:17.400106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e767005a2e4158b004bf85270b1689'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.407310 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e767005a2e4158b004bf85270b1689'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.407966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e767005a2e4158b004bf85270b1689'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.408574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e767005a2e4158b004bf85270b1689'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.416280 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a3f3ef29-561a-4e55-a4a6-66dfef81b566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e767005a2e4158b004bf85270b1689'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.434694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1323/2637] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:17 2022] DELETE /identity/v3/users/85e767005a2e4158b004bf85270b1689 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:17.716034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:17.717484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:17.733282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c333b71-5630-40fc-9e0c-dc97c03cc12b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:17.791175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:17.791822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:17.792103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:17.792377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] PATH_INFO: `/v3/projects/eaeebc93abfb462b86fd6b64291d78a7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:17.793526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaeebc93abfb462b86fd6b64291d78a7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:17.795505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:17.841338 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaeebc93abfb462b86fd6b64291d78a7', 'request_id': 'req-b0c4e75c-38fd-476e-bbb9-ee525e54245e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.976055 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaeebc93abfb462b86fd6b64291d78a7', 'request_id': 'req-b0c4e75c-38fd-476e-bbb9-ee525e54245e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.982352 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaeebc93abfb462b86fd6b64291d78a7', 'request_id': 'req-b0c4e75c-38fd-476e-bbb9-ee525e54245e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.998417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] The token cache is being invalidate because project eaeebc93abfb462b86fd6b64291d78a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:17.998704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:17.999360 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b0c4e75c-38fd-476e-bbb9-ee525e54245e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:18.001491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1315/2638] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:17 2022] DELETE /identity/v3/projects/eaeebc93abfb462b86fd6b64291d78a7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:18.006669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da011f8-e366-42bb-9980-f4064ee939c0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.007214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da011f8-e366-42bb-9980-f4064ee939c0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.007446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da011f8-e366-42bb-9980-f4064ee939c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.017051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:18.029107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4da011f8-e366-42bb-9980-f4064ee939c0 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:18.030734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4da011f8-e366-42bb-9980-f4064ee939c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:18.097640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1324/2639] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:18.105893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca56cc3-aaed-4e8b-ba8a-6b356a897630 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.106187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca56cc3-aaed-4e8b-ba8a-6b356a897630 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.106457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca56cc3-aaed-4e8b-ba8a-6b356a897630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.126375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1ca56cc3-aaed-4e8b-ba8a-6b356a897630 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:18.127583 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1ca56cc3-aaed-4e8b-ba8a-6b356a897630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:18.135394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1316/2640] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:18.139460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da011f8-e366-42bb-9980-f4064ee939c0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.149320 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da011f8-e366-42bb-9980-f4064ee939c0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.154032 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.154640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.154954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.155228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.155910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:18.156731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.158417 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.170290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1325/2641] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:18.175155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca56cc3-aaed-4e8b-ba8a-6b356a897630 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.184359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca56cc3-aaed-4e8b-ba8a-6b356a897630 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.188938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.189537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.189808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.190076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.190738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.192619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.207967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:18.222038 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1317/2642] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:18.226033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.236613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c48e1-5d5a-4550-a798-cd637c657a0c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.293434 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.294069 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.294355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.294633 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.295427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.297965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.319528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1326/2643] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:18.323771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.334690 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa795be-1896-49e0-b4d4-7db370cc0987 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.339165 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.339960 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.340279 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.340676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.341587 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.343936 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.353603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1318/2644] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:18.357644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.369329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9864d9-aa14-45e5-ba38-00a361a0ecaa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.374305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26e98d-55d4-431e-9aaf-ee8fe7b37419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.374997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26e98d-55d4-431e-9aaf-ee8fe7b37419 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.375281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26e98d-55d4-431e-9aaf-ee8fe7b37419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.375674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26e98d-55d4-431e-9aaf-ee8fe7b37419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.376364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26e98d-55d4-431e-9aaf-ee8fe7b37419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.377994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26e98d-55d4-431e-9aaf-ee8fe7b37419 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.389876 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1327/2645] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:18.394991 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.405797 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080520e-d651-440f-bf30-94e1c255b760 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.411257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.412103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.412199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.412449 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] PATH_INFO: `/v3/projects/ffdc3b4d3aec48e5bf26a1ec48fb01ec/users/502f21284ff74d68881d53d27c99d434/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.428142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdc3b4d3aec48e5bf26a1ec48fb01ec, user_id=502f21284ff74d68881d53d27c99d434, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.430765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.446040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1319/2646] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:18 2022] PUT /identity/v3/projects/ffdc3b4d3aec48e5bf26a1ec48fb01ec/users/502f21284ff74d68881d53d27c99d434/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:18.456329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc54b-cc52-40d0-9451-aeafc7bd89fc None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.456620 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc54b-cc52-40d0-9451-aeafc7bd89fc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.456894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc54b-cc52-40d0-9451-aeafc7bd89fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.485970 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-054bc54b-cc52-40d0-9451-aeafc7bd89fc None None] MFA Rules not processed for user `502f21284ff74d68881d53d27c99d434`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:18.487400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-054bc54b-cc52-40d0-9451-aeafc7bd89fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:18.551687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:18.556951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1328/2647] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:18.560924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.625877 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad563f-96f5-4f76-8eba-98aa7dde5c0f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.630590 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.631265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.631688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.631964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.632623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.634499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.662198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1320/2648] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:18.666441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bc54b-cc52-40d0-9451-aeafc7bd89fc None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.678005 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bc54b-cc52-40d0-9451-aeafc7bd89fc None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.737481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.738139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.738421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.738694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.739443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.741341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.766838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1329/2649] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:18.768275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.778661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766149f-cc8c-456f-8309-e4e0701ad4c7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.783746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.784365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.784648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.784931 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.785653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.787316 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.798347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1321/2650] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:18.802841 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.813669 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576aa0-ba8b-4e4c-9564-88c5b93a9b76 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.818357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.819080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.819360 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.819623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] PATH_INFO: `/v3/projects/9c8d96ceee5344889c2e59f247b58608/users/5ee4fb71af594712b99e2abe945bb2db/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.835276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8d96ceee5344889c2e59f247b58608, user_id=5ee4fb71af594712b99e2abe945bb2db, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.837913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.856327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.856935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1330/2651] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:18 2022] PUT /identity/v3/projects/9c8d96ceee5344889c2e59f247b58608/users/5ee4fb71af594712b99e2abe945bb2db/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:18.867102 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238e59-1fa7-40a0-a9db-e4347723f658 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.926281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18100a78-0f4b-4243-9021-e7d067f25a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.926997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18100a78-0f4b-4243-9021-e7d067f25a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.927275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18100a78-0f4b-4243-9021-e7d067f25a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.927544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18100a78-0f4b-4243-9021-e7d067f25a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.928223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18100a78-0f4b-4243-9021-e7d067f25a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.929770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18100a78-0f4b-4243-9021-e7d067f25a43 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.939700 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1322/2652] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:18.943870 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:18.953996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66238-6d04-4191-8fb7-3d03541cb715 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:18.958628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:18.959311 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.959598 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.959872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] PATH_INFO: `/v3/projects/9c8d96ceee5344889c2e59f247b58608/users/5ee4fb71af594712b99e2abe945bb2db/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:18.961820 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8d96ceee5344889c2e59f247b58608, user_id=5ee4fb71af594712b99e2abe945bb2db, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:18.964466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:18.975118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1331/2653] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:18 2022] PUT /identity/v3/projects/9c8d96ceee5344889c2e59f247b58608/users/5ee4fb71af594712b99e2abe945bb2db/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:18.989262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739be3aa-1833-46b5-9104-3e9277e34bfa None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:18.989470 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739be3aa-1833-46b5-9104-3e9277e34bfa None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:18.989744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739be3aa-1833-46b5-9104-3e9277e34bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:19.017571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-739be3aa-1833-46b5-9104-3e9277e34bfa None None] MFA Rules not processed for user `5ee4fb71af594712b99e2abe945bb2db`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:19.018776 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-739be3aa-1833-46b5-9104-3e9277e34bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:19.037471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:19.087581 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1323/2654] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:19.100978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:19.103036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:19.166860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abdc95-6cbb-4bb0-9f72-05ee41d978c2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:19.170656 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:19.171313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:19.171599 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:19.172507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:19.181611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:19.183362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:19.192810 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1332/2655] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:19.283569 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739be3aa-1833-46b5-9104-3e9277e34bfa None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:19.285045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:19.286780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:19.291612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739be3aa-1833-46b5-9104-3e9277e34bfa None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:19.295244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cabfa3-3c1f-4d12-a76f-9b8fc1d6cfed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:19.295890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cabfa3-3c1f-4d12-a76f-9b8fc1d6cfed service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:19.296266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cabfa3-3c1f-4d12-a76f-9b8fc1d6cfed service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:19.296619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cabfa3-3c1f-4d12-a76f-9b8fc1d6cfed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:19.347772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:19.353212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb853a-edd3-4361-8379-3377c199beba service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:19.357308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cabfa3-3c1f-4d12-a76f-9b8fc1d6cfed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:19.358732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cabfa3-3c1f-4d12-a76f-9b8fc1d6cfed service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:19.359342 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:19.360056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:19.360408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:19.360752 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] PATH_INFO: `/v3/users/a4fdb24d6be14cfb9f9a673e48187c21` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:19.361990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4fdb24d6be14cfb9f9a673e48187c21)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:19.364113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:19.367643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1324/2656] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:19.411307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4fdb24d6be14cfb9f9a673e48187c21'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:19.418350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4fdb24d6be14cfb9f9a673e48187c21'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:19.419056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4fdb24d6be14cfb9f9a673e48187c21'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:19.419687 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4fdb24d6be14cfb9f9a673e48187c21'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:19.427370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4fdb24d6be14cfb9f9a673e48187c21'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:19.448261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1333/2657] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:19 2022] DELETE /identity/v3/users/a4fdb24d6be14cfb9f9a673e48187c21 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:19.453806 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:19.454094 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:19.454376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:19.477361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:19.479339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:19.546578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1325/2658] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:19.556566 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:19.558003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:19.618838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1868d3-10af-4ee7-9ec3-4580d69b55d7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:19.622553 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:19.623205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:19.623487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:19.623760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:19.633166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:19.634591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:19.644545 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1334/2659] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:19.912456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:19.913928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:19.926134 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f149b8e-b8f0-4ed7-a9e6-cb3dd18ae63d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:19.942258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:19.942992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:19.943275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:19.943547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] PATH_INFO: `/v3/projects/4ad53e497b9940d7b95422bdbb8f9899` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:19.944762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad53e497b9940d7b95422bdbb8f9899)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:19.946660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:19.997753 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad53e497b9940d7b95422bdbb8f9899', 'request_id': 'req-9bc31631-6233-4882-af87-38a373eaf7db'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:19.999090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:20.006687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad53e497b9940d7b95422bdbb8f9899', 'request_id': 'req-9bc31631-6233-4882-af87-38a373eaf7db'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.012578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad53e497b9940d7b95422bdbb8f9899', 'request_id': 'req-9bc31631-6233-4882-af87-38a373eaf7db'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.025346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] The token cache is being invalidate because project 4ad53e497b9940d7b95422bdbb8f9899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:20.025699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.026399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.028103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1326/2660] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:19 2022] DELETE /identity/v3/projects/4ad53e497b9940d7b95422bdbb8f9899 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:20.032177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:20.044374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2ff31-630f-4435-91b6-fe458244352d service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:20.103167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:20.103817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.103999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.104279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] PATH_INFO: `/v3/users/bcd85f1f0e594404bfeac3b1cfcf5eb1` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.105511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd85f1f0e594404bfeac3b1cfcf5eb1)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:20.107429 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:20.131880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:20.156987 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd85f1f0e594404bfeac3b1cfcf5eb1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.163726 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd85f1f0e594404bfeac3b1cfcf5eb1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.164394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd85f1f0e594404bfeac3b1cfcf5eb1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.165117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd85f1f0e594404bfeac3b1cfcf5eb1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.172471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3336256f-d187-4d8a-bb6c-57ecfc191d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd85f1f0e594404bfeac3b1cfcf5eb1'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.190314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1335/2661] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:20 2022] DELETE /identity/v3/users/bcd85f1f0e594404bfeac3b1cfcf5eb1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:20.460334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:20.461760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:20.477694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc31631-6233-4882-af87-38a373eaf7db admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:20.535443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:20.536089 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.536371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.536699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] PATH_INFO: `/v3/projects/ee2c0c416935435d93fe2a46f528a2fa` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.537859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee2c0c416935435d93fe2a46f528a2fa)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:20.539767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:20.586164 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee2c0c416935435d93fe2a46f528a2fa', 'request_id': 'req-feaa8222-9ccc-4957-9fc8-42491a36fe48'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.592189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee2c0c416935435d93fe2a46f528a2fa', 'request_id': 'req-feaa8222-9ccc-4957-9fc8-42491a36fe48'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.598646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee2c0c416935435d93fe2a46f528a2fa', 'request_id': 'req-feaa8222-9ccc-4957-9fc8-42491a36fe48'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.613771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] The token cache is being invalidate because project ee2c0c416935435d93fe2a46f528a2fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:20.614056 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.614710 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-feaa8222-9ccc-4957-9fc8-42491a36fe48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:20.616303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1327/2662] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:20 2022] DELETE /identity/v3/projects/ee2c0c416935435d93fe2a46f528a2fa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:20.621166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067196-4d0c-4041-9e04-f5c0ee91281f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.621533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067196-4d0c-4041-9e04-f5c0ee91281f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.621912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067196-4d0c-4041-9e04-f5c0ee91281f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.642394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-04067196-4d0c-4041-9e04-f5c0ee91281f None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:20.643653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-04067196-4d0c-4041-9e04-f5c0ee91281f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:20.704721 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1336/2663] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:20.714220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f0ae-0f85-4872-99e6-6da621e554fb None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.714508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f0ae-0f85-4872-99e6-6da621e554fb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.714780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f0ae-0f85-4872-99e6-6da621e554fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.735588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ed22f0ae-0f85-4872-99e6-6da621e554fb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:20.737028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ed22f0ae-0f85-4872-99e6-6da621e554fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:20.745079 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1328/2664] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:20.749421 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067196-4d0c-4041-9e04-f5c0ee91281f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:20.759314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067196-4d0c-4041-9e04-f5c0ee91281f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:20.764188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:20.764767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.765046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.765835 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.766529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:20.766851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:20.768441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:20.784537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1337/2665] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:20.785904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22f0ae-0f85-4872-99e6-6da621e554fb None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:20.795821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22f0ae-0f85-4872-99e6-6da621e554fb None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:20.801390 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:20.802012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.802356 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.802695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.803465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:20.806916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:20.840051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1329/2666] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:20 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:20.841238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:20.857174 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9880c-cbb7-416b-9a66-2aa021160226 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:20.927936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:20.928556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.928833 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.929104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.929782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:20.931758 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:20.952514 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1338/2667] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:20 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:20.957100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:20.967413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1733a25-3a0d-4f6c-8951-d2400992e058 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:20.972736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:20.973341 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:20.973619 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:20.973892 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:20.974571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:20.976178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:20.986295 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1330/2668] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:20.990324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:21.000331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442ac2-6cf8-43ea-94ff-9824b171d605 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:21.004882 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13aaa2-e3fc-4929-93f8-45005eefddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:21.005481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13aaa2-e3fc-4929-93f8-45005eefddd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:21.005765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13aaa2-e3fc-4929-93f8-45005eefddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:21.006035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13aaa2-e3fc-4929-93f8-45005eefddd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:21.006711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13aaa2-e3fc-4929-93f8-45005eefddd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:21.008331 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13aaa2-e3fc-4929-93f8-45005eefddd6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:21.019386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1339/2669] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:21.023339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:21.033827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b631495-c0db-43d1-8c44-1456c7547933 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:21.038435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:21.039143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:21.039419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:21.039689 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] PATH_INFO: `/v3/projects/20ca336d1b074401b5a534237b320e56/users/5470071a7ff8436695b62355883940cf/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:21.053577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20ca336d1b074401b5a534237b320e56, user_id=5470071a7ff8436695b62355883940cf, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:21.056216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:21.068896 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1331/2670] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:21 2022] PUT /identity/v3/projects/20ca336d1b074401b5a534237b320e56/users/5470071a7ff8436695b62355883940cf/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:21.080448 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:21.080731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:21.081000 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:21.088142 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:21.116533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] MFA Rules not processed for user `5470071a7ff8436695b62355883940cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:21.116533 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:21.184986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1340/2671] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:21.195065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:21.196744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:21.203663 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:21.256890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79050e8-bb35-4fcc-85fb-e58592f322d3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:21.260667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675edbb8-d79e-44f2-a742-a26b63cc588a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:21.261291 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675edbb8-d79e-44f2-a742-a26b63cc588a service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:21.261573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675edbb8-d79e-44f2-a742-a26b63cc588a service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:21.261871 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675edbb8-d79e-44f2-a742-a26b63cc588a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:21.270882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675edbb8-d79e-44f2-a742-a26b63cc588a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:21.272301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675edbb8-d79e-44f2-a742-a26b63cc588a service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:21.281287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1332/2672] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:31.177814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=31 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:31.277040 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=24 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:33.346511 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:33.347979 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:33.410672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea27576-049f-49e5-b9cb-ffaed1cbfd0a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:33.415276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:33.415909 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:33.416189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:33.416464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] PATH_INFO: `/v3/users/502f21284ff74d68881d53d27c99d434` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:33.417650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=502f21284ff74d68881d53d27c99d434)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:33.419672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:33.463951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502f21284ff74d68881d53d27c99d434'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:33.471006 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502f21284ff74d68881d53d27c99d434'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:33.471649 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502f21284ff74d68881d53d27c99d434'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:33.472247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502f21284ff74d68881d53d27c99d434'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:33.479196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502f21284ff74d68881d53d27c99d434'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:33.496788 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1341/2673] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:33 2022] DELETE /identity/v3/users/502f21284ff74d68881d53d27c99d434 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:33.501331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:33.501805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:33.501932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:33.520758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:33.521942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:33.579708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1333/2674] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:33.589224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:33.590605 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:33.646663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c352571-bbb5-4751-998a-b8941b3f3ee8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:33.650363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:33.651004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:33.651285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:33.651558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:33.661195 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:33.662614 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:33.671904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1342/2675] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:33.939319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:33.940833 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:33.954253 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a52c9-94fc-4b88-882a-6d80d45375e9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:33.961488 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:33.962142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:33.962420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:33.962694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] PATH_INFO: `/v3/projects/ffdc3b4d3aec48e5bf26a1ec48fb01ec` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:33.963879 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffdc3b4d3aec48e5bf26a1ec48fb01ec)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:33.965767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:34.013113 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffdc3b4d3aec48e5bf26a1ec48fb01ec', 'request_id': 'req-e872930c-591c-485d-b43a-86c6f8254479'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.019855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffdc3b4d3aec48e5bf26a1ec48fb01ec', 'request_id': 'req-e872930c-591c-485d-b43a-86c6f8254479'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.025477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffdc3b4d3aec48e5bf26a1ec48fb01ec', 'request_id': 'req-e872930c-591c-485d-b43a-86c6f8254479'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.037189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] The token cache is being invalidate because project ffdc3b4d3aec48e5bf26a1ec48fb01ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:34.037471 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.038078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.039597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1334/2676] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:33 2022] DELETE /identity/v3/projects/ffdc3b4d3aec48e5bf26a1ec48fb01ec => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:34.043860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:34.053509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186151ca-082c-4be8-9b83-f66472fd0cf3 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:34.071500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:34.109980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:34.110608 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.110927 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.111223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] PATH_INFO: `/v3/users/5ee4fb71af594712b99e2abe945bb2db` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.112384 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee4fb71af594712b99e2abe945bb2db)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:34.114304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:34.158723 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee4fb71af594712b99e2abe945bb2db'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.165520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee4fb71af594712b99e2abe945bb2db'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.166162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee4fb71af594712b99e2abe945bb2db'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.166755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee4fb71af594712b99e2abe945bb2db'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.173965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f4c0060-4c81-4b04-9cea-a241365a82d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee4fb71af594712b99e2abe945bb2db'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.192192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1343/2677] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:34 2022] DELETE /identity/v3/users/5ee4fb71af594712b99e2abe945bb2db => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:34.463049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:34.464582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:34.478393 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872930c-591c-485d-b43a-86c6f8254479 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:34.534461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:34.535161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.535327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.535603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] PATH_INFO: `/v3/projects/9c8d96ceee5344889c2e59f247b58608` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.536747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c8d96ceee5344889c2e59f247b58608)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:34.538618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:34.568500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:34.589350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c8d96ceee5344889c2e59f247b58608', 'request_id': 'req-54716486-18ee-4ce0-87f7-86fff416867b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.596705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c8d96ceee5344889c2e59f247b58608', 'request_id': 'req-54716486-18ee-4ce0-87f7-86fff416867b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.602131 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c8d96ceee5344889c2e59f247b58608', 'request_id': 'req-54716486-18ee-4ce0-87f7-86fff416867b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.616445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] The token cache is being invalidate because project 9c8d96ceee5344889c2e59f247b58608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:34.616744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.617375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54716486-18ee-4ce0-87f7-86fff416867b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:34.618924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1335/2678] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:34 2022] DELETE /identity/v3/projects/9c8d96ceee5344889c2e59f247b58608 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:34.626574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac98f5-c6bc-4b21-952c-0575bf55cf37 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.627166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac98f5-c6bc-4b21-952c-0575bf55cf37 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.627166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac98f5-c6bc-4b21-952c-0575bf55cf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.649073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-87ac98f5-c6bc-4b21-952c-0575bf55cf37 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:34.650287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-87ac98f5-c6bc-4b21-952c-0575bf55cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:34.711905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1344/2679] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:34.719890 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91df71-6b64-45cc-955b-e4103f5b56c6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.720185 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91df71-6b64-45cc-955b-e4103f5b56c6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.720464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91df71-6b64-45cc-955b-e4103f5b56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.739373 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-db91df71-6b64-45cc-955b-e4103f5b56c6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:34.740571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-db91df71-6b64-45cc-955b-e4103f5b56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:34.748374 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1336/2680] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:34.752960 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac98f5-c6bc-4b21-952c-0575bf55cf37 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:34.762504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac98f5-c6bc-4b21-952c-0575bf55cf37 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:34.767654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:34.768266 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.768543 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.768814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.769499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:34.769823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:34.771469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:34.782790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1345/2681] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:34.786981 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91df71-6b64-45cc-955b-e4103f5b56c6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:34.796047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91df71-6b64-45cc-955b-e4103f5b56c6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:34.800722 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:34.801324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.801607 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.801904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.802563 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:34.804474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:34.832333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1337/2682] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:34 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:34.836412 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:34.847065 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bddb4-91b0-4eb3-b6b1-a27ceff26815 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:34.873204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:34.905816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:34.906428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.906709 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.907016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.907683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:34.909581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:34.929238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1346/2683] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:34 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:34.934394 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:34.947402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b6f8a-6715-4227-9dfd-57238c5bd207 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:34.952270 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:34.953649 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.953921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.954214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.954926 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:34.956494 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:34.966511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1338/2684] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:34.970733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:34.980755 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0d9d-3658-4ab5-bc29-1eac6357a872 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:34.985867 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64dd6f-6138-46dd-9799-f8bfe6efaa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:34.986469 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64dd6f-6138-46dd-9799-f8bfe6efaa82 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:34.986749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64dd6f-6138-46dd-9799-f8bfe6efaa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:34.987055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64dd6f-6138-46dd-9799-f8bfe6efaa82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:34.987740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64dd6f-6138-46dd-9799-f8bfe6efaa82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:34.989321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64dd6f-6138-46dd-9799-f8bfe6efaa82 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.000782 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1347/2685] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:35.004532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.014651 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dae29-75e9-495a-8e25-9f92214c34a1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.020263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.020944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.021209 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.021481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] PATH_INFO: `/v3/projects/ef9c11ca4ba2459aaf75f384c97489fa/users/9466515b104f4dbab8d18289844e40cc/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.036416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9c11ca4ba2459aaf75f384c97489fa, user_id=9466515b104f4dbab8d18289844e40cc, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.039127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.052813 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1339/2686] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:35 2022] PUT /identity/v3/projects/ef9c11ca4ba2459aaf75f384c97489fa/users/9466515b104f4dbab8d18289844e40cc/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:35.065468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb794f6-0e9a-4b98-9288-395d2daa5893 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.065468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb794f6-0e9a-4b98-9288-395d2daa5893 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.065674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb794f6-0e9a-4b98-9288-395d2daa5893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.092339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3fb794f6-0e9a-4b98-9288-395d2daa5893 None None] MFA Rules not processed for user `9466515b104f4dbab8d18289844e40cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:35.093550 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3fb794f6-0e9a-4b98-9288-395d2daa5893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:35.155406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1348/2687] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:35.160114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.222452 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4bb2-79aa-4786-bef5-32357839980c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.227331 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.227948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.228232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.228505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.229160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.231076 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.259154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1340/2688] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:35 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:35.263383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb794f6-0e9a-4b98-9288-395d2daa5893 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.274019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb794f6-0e9a-4b98-9288-395d2daa5893 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.329329 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.330262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.330581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.330996 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.331662 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.333561 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.356267 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1349/2689] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:35 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:35.360752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.370919 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65f0c0-1e77-4cda-bfea-bba00d02c41a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.376249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.376855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.377128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.377398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.378135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.379793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.390124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1341/2690] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:35.394084 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.403902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e092d65-2292-4036-84f8-eac4fd8c2cbe admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.409164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.409852 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.410121 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.410395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] PATH_INFO: `/v3/projects/d8077219e2ab44e89ee036e5a69ad24a/users/774e21a78fed46998853ad695fd6b829/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.425239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8077219e2ab44e89ee036e5a69ad24a, user_id=774e21a78fed46998853ad695fd6b829, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.428279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.440507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1350/2691] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:35 2022] PUT /identity/v3/projects/d8077219e2ab44e89ee036e5a69ad24a/users/774e21a78fed46998853ad695fd6b829/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:35.444444 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.454032 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cccf1-c035-452e-9890-925286b0432d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.474643 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:35.512083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1afcd-5caa-4114-baa9-0fde6c71667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.512701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1afcd-5caa-4114-baa9-0fde6c71667b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.512984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1afcd-5caa-4114-baa9-0fde6c71667b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.513262 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1afcd-5caa-4114-baa9-0fde6c71667b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.513952 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1afcd-5caa-4114-baa9-0fde6c71667b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.515561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1afcd-5caa-4114-baa9-0fde6c71667b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.525023 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1342/2692] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:35.529458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.535401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:35.542047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5670282-921e-4ba6-8262-05ba45e4aa41 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.547502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.548154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.548433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.548708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] PATH_INFO: `/v3/projects/d8077219e2ab44e89ee036e5a69ad24a/users/774e21a78fed46998853ad695fd6b829/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.550670 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8077219e2ab44e89ee036e5a69ad24a, user_id=774e21a78fed46998853ad695fd6b829, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.553326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.563673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1351/2693] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:35 2022] PUT /identity/v3/projects/d8077219e2ab44e89ee036e5a69ad24a/users/774e21a78fed46998853ad695fd6b829/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:35.573761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e129e2-779d-4ece-9ebd-50b9f992b220 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.574225 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e129e2-779d-4ece-9ebd-50b9f992b220 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.574324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e129e2-779d-4ece-9ebd-50b9f992b220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.601384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-43e129e2-779d-4ece-9ebd-50b9f992b220 None None] MFA Rules not processed for user `774e21a78fed46998853ad695fd6b829`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:35.602579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-43e129e2-779d-4ece-9ebd-50b9f992b220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:35.673762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1343/2694] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:35.684952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:35.686712 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:35.743283 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214eba2c-efc8-4e0f-abe7-398e2225fdbe admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:35.746907 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db391965-1484-4b62-bc32-577aaf06c3d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:35.747517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db391965-1484-4b62-bc32-577aaf06c3d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:35.747801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db391965-1484-4b62-bc32-577aaf06c3d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:35.748078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db391965-1484-4b62-bc32-577aaf06c3d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:35.802983 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db391965-1484-4b62-bc32-577aaf06c3d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:35.804439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db391965-1484-4b62-bc32-577aaf06c3d2 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:35.813583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1352/2695] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:40.448656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e129e2-779d-4ece-9ebd-50b9f992b220 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:40.509000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e129e2-779d-4ece-9ebd-50b9f992b220 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:40.513680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:40.514324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:40.514602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:40.514893 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] PATH_INFO: `/v3/users/9466515b104f4dbab8d18289844e40cc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:40.516104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9466515b104f4dbab8d18289844e40cc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:40.517999 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:40.563611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9466515b104f4dbab8d18289844e40cc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:40.564263 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9466515b104f4dbab8d18289844e40cc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:40.564864 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9466515b104f4dbab8d18289844e40cc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:40.565982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:40.578037 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9466515b104f4dbab8d18289844e40cc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:40.584656 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9466515b104f4dbab8d18289844e40cc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:40.597313 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1344/2696] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:40 2022] DELETE /identity/v3/users/9466515b104f4dbab8d18289844e40cc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:40.601805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:40.602290 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:40.602389 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:40.622737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:40.623966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:40.688110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1353/2697] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:40.697815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:40.699284 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:40.768883 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be81d8-5b23-450c-8f42-af4daeab3558 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:40.770598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:40.771254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:40.771533 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:40.771858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:40.783396 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:40.784983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:40.795685 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1345/2698] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:41.106176 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:41.107679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:41.122223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f00955-cd47-4b20-961b-7570e31941ed None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:41.127678 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:41.128453 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:41.128761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:41.129060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] PATH_INFO: `/v3/projects/ef9c11ca4ba2459aaf75f384c97489fa` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:41.130366 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9c11ca4ba2459aaf75f384c97489fa)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:41.132338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:41.175751 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:41.185898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9c11ca4ba2459aaf75f384c97489fa', 'request_id': 'req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.193488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9c11ca4ba2459aaf75f384c97489fa', 'request_id': 'req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.201740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9c11ca4ba2459aaf75f384c97489fa', 'request_id': 'req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.212157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] The token cache is being invalidate because project ef9c11ca4ba2459aaf75f384c97489fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:41.212442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.213062 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.214572 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1354/2699] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:41 2022] DELETE /identity/v3/projects/ef9c11ca4ba2459aaf75f384c97489fa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:41.218441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:41.228214 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249d9de-8039-4234-8023-5ea0e198010c service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:41.283222 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:41.283861 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:41.284141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:41.284415 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] PATH_INFO: `/v3/users/774e21a78fed46998853ad695fd6b829` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:41.285592 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774e21a78fed46998853ad695fd6b829)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:41.287518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:41.332962 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774e21a78fed46998853ad695fd6b829'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.333595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774e21a78fed46998853ad695fd6b829'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.334196 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774e21a78fed46998853ad695fd6b829'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.346154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774e21a78fed46998853ad695fd6b829'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.350429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:41.354052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-12a92df5-1848-4847-98f7-e3a77aa4a5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774e21a78fed46998853ad695fd6b829'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.368059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1346/2700] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:41 2022] DELETE /identity/v3/users/774e21a78fed46998853ad695fd6b829 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:41.648203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:41.649648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:41.664517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61868d8-dd53-47a6-8a04-9ce5811d0eb2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:41.720531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:41.721097 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:41.721359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:41.721637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] PATH_INFO: `/v3/projects/d8077219e2ab44e89ee036e5a69ad24a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:41.722765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8077219e2ab44e89ee036e5a69ad24a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:41.724679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:41.768241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8077219e2ab44e89ee036e5a69ad24a', 'request_id': 'req-952b579f-2866-46f8-afec-d58c120f56d5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.774496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8077219e2ab44e89ee036e5a69ad24a', 'request_id': 'req-952b579f-2866-46f8-afec-d58c120f56d5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.782335 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8077219e2ab44e89ee036e5a69ad24a', 'request_id': 'req-952b579f-2866-46f8-afec-d58c120f56d5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.792433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] The token cache is being invalidate because project d8077219e2ab44e89ee036e5a69ad24a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:41.792703 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.793334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-952b579f-2866-46f8-afec-d58c120f56d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:41.794846 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1355/2701] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:41 2022] DELETE /identity/v3/projects/d8077219e2ab44e89ee036e5a69ad24a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:41.799680 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f99648-6bdb-4364-b81c-e2b24d6961b7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:41.800143 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f99648-6bdb-4364-b81c-e2b24d6961b7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:41.800242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f99648-6bdb-4364-b81c-e2b24d6961b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:41.819448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c8f99648-6bdb-4364-b81c-e2b24d6961b7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:41.820639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c8f99648-6bdb-4364-b81c-e2b24d6961b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:41.882717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1347/2702] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:41.890489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db821-6bb1-4b68-9cd4-e227a171e12d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:41.891312 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db821-6bb1-4b68-9cd4-e227a171e12d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:41.891593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db821-6bb1-4b68-9cd4-e227a171e12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:41.911992 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-447db821-6bb1-4b68-9cd4-e227a171e12d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:41.913477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-447db821-6bb1-4b68-9cd4-e227a171e12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:41.921465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1356/2703] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:41.925857 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f99648-6bdb-4364-b81c-e2b24d6961b7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:41.935695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f99648-6bdb-4364-b81c-e2b24d6961b7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:41.940141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:41.940736 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:41.941013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:41.941282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:41.941982 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:41.942282 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:41.943889 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:41.956287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1348/2704] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:41.961094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db821-6bb1-4b68-9cd4-e227a171e12d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:41.971117 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db821-6bb1-4b68-9cd4-e227a171e12d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:41.975701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:41.976304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:41.976582 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:41.976853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:41.977505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:41.979430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:42.006475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1357/2705] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:41 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:42.010795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:42.021638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4d71f-1451-4253-a166-79bca5ae7525 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:42.077567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:42.078187 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:42.079074 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:42.079357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:42.080020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:42.081909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:42.103787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1349/2706] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:42 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:42.108067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:42.118233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17258f7-6935-4eca-a887-dc157dcf2faa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:42.123211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:42.123818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:42.124103 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:42.124375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:42.125624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:42.127387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:42.137697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1358/2707] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:42.142230 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:42.152848 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44103d4-2ada-4dd9-acb4-d7fb773e7f22 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:42.158402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ca222-3ab7-4bbb-bbe3-049d446005d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:42.159168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ca222-3ab7-4bbb-bbe3-049d446005d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:42.159540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ca222-3ab7-4bbb-bbe3-049d446005d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:42.159823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ca222-3ab7-4bbb-bbe3-049d446005d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:42.160631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ca222-3ab7-4bbb-bbe3-049d446005d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:42.162617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ca222-3ab7-4bbb-bbe3-049d446005d2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:42.174807 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1350/2708] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:42.179285 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:42.190953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a8df5-4b6f-45c4-8f3c-f7dd0a45b9c1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:42.195600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:42.196254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:42.196528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:42.196799 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] PATH_INFO: `/v3/projects/4bd221dfff634404a071ad30db96b99d/users/418f616114fa430cbc73d8cbe3263c52/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:42.215878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd221dfff634404a071ad30db96b99d, user_id=418f616114fa430cbc73d8cbe3263c52, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:42.219500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:42.222394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:42.239190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1359/2709] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:42 2022] PUT /identity/v3/projects/4bd221dfff634404a071ad30db96b99d/users/418f616114fa430cbc73d8cbe3263c52/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:42.244751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b8f70-c68b-47ed-9ea4-48b34ebe9b96 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:42.245117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b8f70-c68b-47ed-9ea4-48b34ebe9b96 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:42.245402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b8f70-c68b-47ed-9ea4-48b34ebe9b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:42.270297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-627b8f70-c68b-47ed-9ea4-48b34ebe9b96 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:42.271715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-627b8f70-c68b-47ed-9ea4-48b34ebe9b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:42.325192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:42.340525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1351/2710] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:42.350378 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:42.352119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:42.409585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9097a-fbbb-4879-a7b1-c0415fc57d3e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:42.413942 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:42.414554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:42.414827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:42.415130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:42.424271 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:42.425995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:42.437302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1360/2711] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:43.052216 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:43.052739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:43.052739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:43.081153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] MFA Rules not processed for user `418f616114fa430cbc73d8cbe3263c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:43.082611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:43.148830 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1352/2712] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:43.160306 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:43.161700 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:43.219965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f02c9-a31c-4066-8837-03e458e596bb service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:43.223781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d1010-f3e7-451a-8de8-7692e605384b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:43.224546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d1010-f3e7-451a-8de8-7692e605384b service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:43.224895 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d1010-f3e7-451a-8de8-7692e605384b service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:43.225243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d1010-f3e7-451a-8de8-7692e605384b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:43.233794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d1010-f3e7-451a-8de8-7692e605384b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:43.235248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d1010-f3e7-451a-8de8-7692e605384b service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:43.243947 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1361/2713] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:43.605375 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:43.607124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:43.620737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867ad79-ed4d-4800-acf7-28fa3aa78564 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:43.625815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:43.626621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:43.627003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:43.627308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] PATH_INFO: `/v3/users/4db1aaa0d9014e95984b7fe85852e1cb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:43.628472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db1aaa0d9014e95984b7fe85852e1cb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:43.630367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:43.674984 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db1aaa0d9014e95984b7fe85852e1cb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:43.675639 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db1aaa0d9014e95984b7fe85852e1cb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:43.676254 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db1aaa0d9014e95984b7fe85852e1cb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:43.691998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db1aaa0d9014e95984b7fe85852e1cb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:43.699220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db1aaa0d9014e95984b7fe85852e1cb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:43.712684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1353/2714] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:43 2022] DELETE /identity/v3/users/4db1aaa0d9014e95984b7fe85852e1cb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:43.717177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:43.717465 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:43.717744 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:43.736234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:43.737439 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:43.794167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1362/2715] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:43.803717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:43.805112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:43.863548 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe02067-5d65-4a2b-8737-915185487bf0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:43.868184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:43.868814 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:43.869095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:43.869371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:43.880554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:43.881974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:43.886139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:43.895154 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1354/2716] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:44.161184 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:44.162676 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:44.174478 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f599d-b1cb-41e8-a953-7390e1db9b98 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:44.179196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:44.179837 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:44.180113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:44.180383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] PATH_INFO: `/v3/projects/a0a3d1d304fd432997f1577268a7e2ae` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:44.181544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a3d1d304fd432997f1577268a7e2ae)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:44.183731 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:44.223240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:44.238989 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a3d1d304fd432997f1577268a7e2ae', 'request_id': 'req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.245430 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a3d1d304fd432997f1577268a7e2ae', 'request_id': 'req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.254892 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a3d1d304fd432997f1577268a7e2ae', 'request_id': 'req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.265647 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] The token cache is being invalidate because project a0a3d1d304fd432997f1577268a7e2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:44.266424 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.267087 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.268596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1363/2717] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:44 2022] DELETE /identity/v3/projects/a0a3d1d304fd432997f1577268a7e2ae => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:44.272551 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:44.282815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c27a2-fd6f-4aaf-a36e-a16c123c293e service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:44.340044 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:44.340687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:44.340967 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:44.341240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] PATH_INFO: `/v3/users/a0843b785b6145629bc45bc9cbc42873` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:44.342391 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0843b785b6145629bc45bc9cbc42873)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:44.344978 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:44.388948 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0843b785b6145629bc45bc9cbc42873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.389596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0843b785b6145629bc45bc9cbc42873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.390205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0843b785b6145629bc45bc9cbc42873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.402633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0843b785b6145629bc45bc9cbc42873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.409644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a7c9ea9f-b5d9-44d4-8314-a6ef09e4735b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0843b785b6145629bc45bc9cbc42873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.422567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1355/2718] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:44 2022] DELETE /identity/v3/users/a0843b785b6145629bc45bc9cbc42873 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:44.707464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:44.709098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:44.723556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4dc38-12ad-4f23-acd2-bc3626f18c7f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:44.780504 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:44.781135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:44.781542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:44.781705 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] PATH_INFO: `/v3/projects/2af661efad174713bc793f00e60c2285` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:44.782840 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af661efad174713bc793f00e60c2285)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:44.784745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:44.831294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af661efad174713bc793f00e60c2285', 'request_id': 'req-2a37f717-37df-40c3-a4e5-04664ee87c39'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.837369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af661efad174713bc793f00e60c2285', 'request_id': 'req-2a37f717-37df-40c3-a4e5-04664ee87c39'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.845969 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af661efad174713bc793f00e60c2285', 'request_id': 'req-2a37f717-37df-40c3-a4e5-04664ee87c39'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.858641 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] The token cache is being invalidate because project 2af661efad174713bc793f00e60c2285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:44.858955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.860125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2a37f717-37df-40c3-a4e5-04664ee87c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:44.861760 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1364/2719] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:44 2022] DELETE /identity/v3/projects/2af661efad174713bc793f00e60c2285 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:44.867418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5ac7f-aa1a-4b45-81ec-689f207d586d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:44.867699 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5ac7f-aa1a-4b45-81ec-689f207d586d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:44.867970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5ac7f-aa1a-4b45-81ec-689f207d586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:44.887580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a7a5ac7f-aa1a-4b45-81ec-689f207d586d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:44.888759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a7a5ac7f-aa1a-4b45-81ec-689f207d586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:44.946189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1356/2720] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:44.954012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe3a82-1e1c-43c1-8699-5ab49a56e8eb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:44.954293 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe3a82-1e1c-43c1-8699-5ab49a56e8eb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:44.954567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe3a82-1e1c-43c1-8699-5ab49a56e8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:44.973975 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-38fe3a82-1e1c-43c1-8699-5ab49a56e8eb None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:44.975223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-38fe3a82-1e1c-43c1-8699-5ab49a56e8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:44.983241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1365/2721] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:44.988016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5ac7f-aa1a-4b45-81ec-689f207d586d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:44.997612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5ac7f-aa1a-4b45-81ec-689f207d586d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.002755 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.003399 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.003676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.003947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.004661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:45.004953 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.006514 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.018057 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1357/2722] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:45.022368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe3a82-1e1c-43c1-8699-5ab49a56e8eb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.031604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe3a82-1e1c-43c1-8699-5ab49a56e8eb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.036319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.036920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.037199 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.037473 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.038125 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.040027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.067654 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1366/2723] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:45 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:45.071671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.077941 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:45.084007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a01f6-2ab5-49cd-b6d3-1f4f99face77 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.140112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.140720 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.140995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.141265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.141922 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.143916 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.164561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1358/2724] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:45 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:45.168844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.179674 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ec38d-0406-4817-b740-c18126626355 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.184508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.185106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.185383 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.185653 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.186332 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.187934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.197518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1367/2725] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:45.201357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.211206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee016d29-bce5-40e6-8cdb-fb8516013386 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.215811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e424f9f-a7a6-4f81-8bff-d7540e62bb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.216430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424f9f-a7a6-4f81-8bff-d7540e62bb00 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.216709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424f9f-a7a6-4f81-8bff-d7540e62bb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.216980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424f9f-a7a6-4f81-8bff-d7540e62bb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.217658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e424f9f-a7a6-4f81-8bff-d7540e62bb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.219756 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e424f9f-a7a6-4f81-8bff-d7540e62bb00 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.230060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1359/2726] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:45.233900 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.244390 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5322e-ba45-46f2-b568-1140d9beeef2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.248949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.249593 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.249868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.250220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] PATH_INFO: `/v3/projects/db103123b05a42a9828cfac3e9222930/users/a08853239b8f4486bb64139958495a81/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.261014 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:45.265650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db103123b05a42a9828cfac3e9222930, user_id=a08853239b8f4486bb64139958495a81, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.269459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.282745 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1368/2727] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:45 2022] PUT /identity/v3/projects/db103123b05a42a9828cfac3e9222930/users/a08853239b8f4486bb64139958495a81/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:45.292985 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc037d-4f43-48ba-ac8d-c5b0eb361719 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.293443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc037d-4f43-48ba-ac8d-c5b0eb361719 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.293561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc037d-4f43-48ba-ac8d-c5b0eb361719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.320644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e0fc037d-4f43-48ba-ac8d-c5b0eb361719 None None] MFA Rules not processed for user `a08853239b8f4486bb64139958495a81`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:45.321836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e0fc037d-4f43-48ba-ac8d-c5b0eb361719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:45.385718 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1360/2728] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:45.390027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.449466 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fbd09-30f1-4ba1-ae6f-586eab491e7a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.454544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.455179 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.455457 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.455727 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.456377 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.458337 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.483914 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1369/2729] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:45 2022] POST /identity/v3/projects => generated 369 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:45.487956 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc037d-4f43-48ba-ac8d-c5b0eb361719 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.498260 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc037d-4f43-48ba-ac8d-c5b0eb361719 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.554613 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.555246 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.555526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.555796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.556453 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.558333 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.579098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1361/2730] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:45 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:45.583210 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.593183 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692eff48-8a5e-493d-ad81-72f54dd64435 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.598452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.599029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.599307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.599579 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.600251 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.601821 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.613981 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1370/2731] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:45.618264 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.623443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:45.631117 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2548c-0b2e-4536-9115-902b18edcfc5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.636288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.636990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.637236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.637507 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] PATH_INFO: `/v3/projects/d69a6f49385c40c286da71ffe0fbac8c/users/b5d2ad310e3f4a3ca5216d841512d4fa/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.652790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69a6f49385c40c286da71ffe0fbac8c, user_id=b5d2ad310e3f4a3ca5216d841512d4fa, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.655505 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.668704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1362/2732] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:45 2022] PUT /identity/v3/projects/d69a6f49385c40c286da71ffe0fbac8c/users/b5d2ad310e3f4a3ca5216d841512d4fa/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:45.672595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.682275 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcf616-c3f2-4f09-8ffb-36c1fa61be91 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.736432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe25689-1b8f-4d01-8e67-8ab4d9c2da2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.737061 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25689-1b8f-4d01-8e67-8ab4d9c2da2c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.737349 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25689-1b8f-4d01-8e67-8ab4d9c2da2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.737630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25689-1b8f-4d01-8e67-8ab4d9c2da2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.738321 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe25689-1b8f-4d01-8e67-8ab4d9c2da2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.740063 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe25689-1b8f-4d01-8e67-8ab4d9c2da2c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.749857 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1371/2733] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:45.754039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.764401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf726528-23ac-4b3e-b150-445dd5ec6420 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.769525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.770182 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.770464 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.770741 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] PATH_INFO: `/v3/projects/d69a6f49385c40c286da71ffe0fbac8c/users/b5d2ad310e3f4a3ca5216d841512d4fa/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.772739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69a6f49385c40c286da71ffe0fbac8c, user_id=b5d2ad310e3f4a3ca5216d841512d4fa, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.775404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:45.786034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1363/2734] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:45 2022] PUT /identity/v3/projects/d69a6f49385c40c286da71ffe0fbac8c/users/b5d2ad310e3f4a3ca5216d841512d4fa/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:45.797100 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.797395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.797673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.824476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] MFA Rules not processed for user `b5d2ad310e3f4a3ca5216d841512d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:45.825682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:45.872562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:45.903767 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1372/2735] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:45.915357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:45.917090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:45.975534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3c286-ca13-4efb-b150-e0c078b12aeb admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:45.979231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7ddef-9879-4481-9e1c-dbe7f2422844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:45.979843 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7ddef-9879-4481-9e1c-dbe7f2422844 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:45.980127 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7ddef-9879-4481-9e1c-dbe7f2422844 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:45.980401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7ddef-9879-4481-9e1c-dbe7f2422844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:45.989549 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7ddef-9879-4481-9e1c-dbe7f2422844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:45.991064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7ddef-9879-4481-9e1c-dbe7f2422844 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:46.001002 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1364/2736] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:46.045733 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:46.047154 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:46.059391 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86a961-2d69-4242-a3d9-8a6d160577d2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:46.063503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:46.064112 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:46.064392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:46.064667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:46.122212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:46.123617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:46.132912 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1373/2737] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:47.777166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480cf7-965a-419d-b1ef-88b974a95125 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:47.777709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480cf7-965a-419d-b1ef-88b974a95125 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:47.777709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480cf7-965a-419d-b1ef-88b974a95125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:47.798298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-89480cf7-965a-419d-b1ef-88b974a95125 None None] MFA Rules not processed for user `418f616114fa430cbc73d8cbe3263c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:47.799520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-89480cf7-965a-419d-b1ef-88b974a95125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:47.879770 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1365/2738] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:48.127773 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:48.129171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:48.185722 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f251b1-0db7-4d61-960e-7d10b099289a service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:48.189382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:48.189999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:48.190276 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:48.190548 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:48.192316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:48.201734 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:48.203145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:48.211048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1374/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:49.452146 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480cf7-965a-419d-b1ef-88b974a95125 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:49.519561 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480cf7-965a-419d-b1ef-88b974a95125 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:49.524783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:49.525431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:49.525712 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:49.525988 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] PATH_INFO: `/v3/users/a08853239b8f4486bb64139958495a81` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:49.527342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08853239b8f4486bb64139958495a81)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:49.529095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:49.547459 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:49.551859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:49.553875 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:49.593474 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08853239b8f4486bb64139958495a81'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:49.595141 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08853239b8f4486bb64139958495a81'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:49.596463 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08853239b8f4486bb64139958495a81'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:49.613297 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08853239b8f4486bb64139958495a81'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:49.619319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c559fd89-deb1-49b1-834c-c80fc9173567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08853239b8f4486bb64139958495a81'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:49.633206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1366/2740] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:49 2022] DELETE /identity/v3/users/a08853239b8f4486bb64139958495a81 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:49.637259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189628d-0177-44db-910e-0e1d7f514f13 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:49.637593 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a1622-031d-486e-8632-c0667586bf3c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:49.637909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a1622-031d-486e-8632-c0667586bf3c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:49.638184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a1622-031d-486e-8632-c0667586bf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:49.658236 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-652a1622-031d-486e-8632-c0667586bf3c None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:49.659476 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-652a1622-031d-486e-8632-c0667586bf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:49.664370 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:49.700164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:49.701092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:49.701386 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:49.701661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:49.704073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:49.716098 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:49.717794 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:49.718266 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1367/2741] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:49.727987 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a1622-031d-486e-8632-c0667586bf3c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:49.728627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1375/2742] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:49.729499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-652a1622-031d-486e-8632-c0667586bf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:49.788574 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a1622-031d-486e-8632-c0667586bf3c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:49.792195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:49.792805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:49.793083 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:49.793354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:49.802805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:49.804257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:49.814670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1368/2743] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:50.099525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:50.101344 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:50.131334 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214597-8630-4a32-922c-88b9dbb31d28 service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:50.136382 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:50.137025 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:50.137299 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:50.137570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] PATH_INFO: `/v3/projects/db103123b05a42a9828cfac3e9222930` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:50.138711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=db103123b05a42a9828cfac3e9222930)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:50.141169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:50.191146 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db103123b05a42a9828cfac3e9222930', 'request_id': 'req-5b1ac171-d44e-4b20-9445-35d2ee032fbd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.197973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db103123b05a42a9828cfac3e9222930', 'request_id': 'req-5b1ac171-d44e-4b20-9445-35d2ee032fbd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.206795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db103123b05a42a9828cfac3e9222930', 'request_id': 'req-5b1ac171-d44e-4b20-9445-35d2ee032fbd'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.223584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] The token cache is being invalidate because project db103123b05a42a9828cfac3e9222930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:50.223872 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.224613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.226541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1376/2744] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:50 2022] DELETE /identity/v3/projects/db103123b05a42a9828cfac3e9222930 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:50.230591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:50.241429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd2cdf-fbac-4cdb-9a91-c1cefd31db1f service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:50.291808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:50.321626 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:50.322274 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:50.323194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:50.323711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] PATH_INFO: `/v3/users/b5d2ad310e3f4a3ca5216d841512d4fa` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:50.325455 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d2ad310e3f4a3ca5216d841512d4fa)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:50.327567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:50.397666 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:50.399196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:50.399327 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d2ad310e3f4a3ca5216d841512d4fa'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.400153 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d2ad310e3f4a3ca5216d841512d4fa'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.400796 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d2ad310e3f4a3ca5216d841512d4fa'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.414178 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ac171-d44e-4b20-9445-35d2ee032fbd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:50.415430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d2ad310e3f4a3ca5216d841512d4fa'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.418973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:50.419611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:50.419883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:50.420157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] PATH_INFO: `/v3/users/5470071a7ff8436695b62355883940cf` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:50.421435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5470071a7ff8436695b62355883940cf)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:50.422709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a12148ec-dcfa-4c70-a112-f70626da367c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d2ad310e3f4a3ca5216d841512d4fa'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.423477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:50.436752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1369/2745] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:50 2022] DELETE /identity/v3/users/b5d2ad310e3f4a3ca5216d841512d4fa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:50.476441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5470071a7ff8436695b62355883940cf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.483784 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5470071a7ff8436695b62355883940cf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.484529 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5470071a7ff8436695b62355883940cf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.485242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5470071a7ff8436695b62355883940cf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.492603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5470071a7ff8436695b62355883940cf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.513749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1377/2746] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:50 2022] DELETE /identity/v3/users/5470071a7ff8436695b62355883940cf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:50.519062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:50.519577 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:50.519694 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:50.538970 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:50.540191 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:50.605928 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1370/2747] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:50.616619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:50.618294 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:50.636248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:50.693351 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fb072-2973-4dce-9a6a-11d52e65833e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:50.699526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce453467-290b-4224-b8fe-6d400c944241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:50.700365 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce453467-290b-4224-b8fe-6d400c944241 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:50.700696 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce453467-290b-4224-b8fe-6d400c944241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:50.700980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce453467-290b-4224-b8fe-6d400c944241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:50.712965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce453467-290b-4224-b8fe-6d400c944241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:50.714503 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce453467-290b-4224-b8fe-6d400c944241 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:50.727823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1378/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:50.814188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:50.816239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:50.837414 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f627b-a322-4c20-818b-6853758c66f6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:50.842251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:50.843340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:50.843558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:50.844647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] PATH_INFO: `/v3/projects/d69a6f49385c40c286da71ffe0fbac8c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:50.845947 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69a6f49385c40c286da71ffe0fbac8c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:50.848007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:50.924035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69a6f49385c40c286da71ffe0fbac8c', 'request_id': 'req-d3187a85-6063-4169-b4ab-1ef2941c14ff'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.926252 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:50.936880 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69a6f49385c40c286da71ffe0fbac8c', 'request_id': 'req-d3187a85-6063-4169-b4ab-1ef2941c14ff'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.958711 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69a6f49385c40c286da71ffe0fbac8c', 'request_id': 'req-d3187a85-6063-4169-b4ab-1ef2941c14ff'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.980361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] The token cache is being invalidate because project d69a6f49385c40c286da71ffe0fbac8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:50.980675 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.981382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d3187a85-6063-4169-b4ab-1ef2941c14ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:50.983491 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1371/2749] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:50 2022] DELETE /identity/v3/projects/d69a6f49385c40c286da71ffe0fbac8c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:50.990261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c485187-6cc3-4f9c-89ad-c195fc6ce101 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:50.990795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c485187-6cc3-4f9c-89ad-c195fc6ce101 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:50.991159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c485187-6cc3-4f9c-89ad-c195fc6ce101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.016281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6c485187-6cc3-4f9c-89ad-c195fc6ce101 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:51.017530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6c485187-6cc3-4f9c-89ad-c195fc6ce101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:51.081182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1379/2750] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:51.089534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb15f3-efb6-40f7-81b0-5c5c43ec9964 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.089534 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb15f3-efb6-40f7-81b0-5c5c43ec9964 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.089783 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb15f3-efb6-40f7-81b0-5c5c43ec9964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.110961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8ccb15f3-efb6-40f7-81b0-5c5c43ec9964 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:51.112275 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8ccb15f3-efb6-40f7-81b0-5c5c43ec9964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:51.120572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1372/2751] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:51.124848 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c485187-6cc3-4f9c-89ad-c195fc6ce101 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.136211 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c485187-6cc3-4f9c-89ad-c195fc6ce101 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.141405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.142020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.142300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.142606 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.143335 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:51.143642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.145229 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.158789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1380/2752] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:51.162735 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb15f3-efb6-40f7-81b0-5c5c43ec9964 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.172683 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb15f3-efb6-40f7-81b0-5c5c43ec9964 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.177281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.177884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.178160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.178430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.179118 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.181086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.208746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1373/2753] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:51.212808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.223307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9159e45-2f0b-44fb-95dd-be547393f1be admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.262202 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.264431 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:51.285998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.286735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.287167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.287510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.288254 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.298756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.329172 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1381/2754] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:51.333204 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.346565 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc6a0-94a1-455e-8318-26f48de4e4d6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.346798 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3029b-afa5-434e-b983-cc28575ce48f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.351245 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.351457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.351853 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.352080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.352189 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.352361 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.352468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.352640 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] PATH_INFO: `/v3/projects/20ca336d1b074401b5a534237b320e56` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.353243 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.353811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20ca336d1b074401b5a534237b320e56)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.354905 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.355727 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.366611 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1382/2755] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:51.370359 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.381157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eafbdd-7c5a-4c1c-b3ca-e2e12f59addb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.385932 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.386546 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.386824 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.387138 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.387880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.389471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.400711 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1383/2756] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:51.404544 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.415319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f9b63-4a3d-40e1-a47d-a255d03cf824 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.418247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20ca336d1b074401b5a534237b320e56', 'request_id': 'req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:51.426034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20ca336d1b074401b5a534237b320e56', 'request_id': 'req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:51.426216 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:51.433774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20ca336d1b074401b5a534237b320e56', 'request_id': 'req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:51.450195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] The token cache is being invalidate because project 20ca336d1b074401b5a534237b320e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:51.450686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:51.451543 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b7dd46a5-8a6e-4574-9c89-20f777fbba6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:51.453791 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1374/2757] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:51 2022] DELETE /identity/v3/projects/20ca336d1b074401b5a534237b320e56 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:51.458387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae391fd-d3a8-453b-a3d1-4320daac3af3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.458942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae391fd-d3a8-453b-a3d1-4320daac3af3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.459387 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae391fd-d3a8-453b-a3d1-4320daac3af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.480760 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1ae391fd-d3a8-453b-a3d1-4320daac3af3 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:51.482365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1ae391fd-d3a8-453b-a3d1-4320daac3af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:51.484603 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916991c9-27b7-4ad4-8631-6c2427904e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.485333 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916991c9-27b7-4ad4-8631-6c2427904e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.485617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916991c9-27b7-4ad4-8631-6c2427904e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.485894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916991c9-27b7-4ad4-8631-6c2427904e8b admin admin] PATH_INFO: `/v3/projects/da1cd6e7ebc748299fb42f6dab6ea5b0/users/3aee99dcfc3a4a54abc74404f09ce8a1/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.491914 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1375/2758] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:51.499654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb93e27-f78e-40f0-9a70-143637efd56b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.500247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb93e27-f78e-40f0-9a70-143637efd56b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.500520 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb93e27-f78e-40f0-9a70-143637efd56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.505597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916991c9-27b7-4ad4-8631-6c2427904e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1cd6e7ebc748299fb42f6dab6ea5b0, user_id=3aee99dcfc3a4a54abc74404f09ce8a1, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.508701 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916991c9-27b7-4ad4-8631-6c2427904e8b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.521306 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fbb93e27-f78e-40f0-9a70-143637efd56b None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:51.522602 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fbb93e27-f78e-40f0-9a70-143637efd56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:51.528045 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1384/2759] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:51 2022] PUT /identity/v3/projects/da1cd6e7ebc748299fb42f6dab6ea5b0/users/3aee99dcfc3a4a54abc74404f09ce8a1/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:51.538948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.539246 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.539615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.560244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:51.573261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] MFA Rules not processed for user `3aee99dcfc3a4a54abc74404f09ce8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:51.574488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:51.617398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1376/2760] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:51.621499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb93e27-f78e-40f0-9a70-143637efd56b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.632557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb93e27-f78e-40f0-9a70-143637efd56b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.637832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.638472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.638749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.639047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.639737 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:51.640043 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.642155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.658901 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1377/2761] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 14:41:51.661597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.673316 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1385/2762] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:51.673876 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030d48-d78c-4050-85ed-17ff4b5e4a94 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.675237 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.677214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:51.677684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.678303 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.678579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.679038 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.679702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.681588 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.692956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e42ec-840f-4ffe-b8d6-5b09143238b5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.697918 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.698542 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.698825 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.699134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.699798 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.701826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.716969 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1378/2763] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:51.717647 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.736878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1386/2764] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:51.741093 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.800771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4e6f1-97ef-49ba-a543-a36b511ce9a7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.826284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43921fc-514b-4c59-8e09-7eb0e83188e1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.832991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.833625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.833957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.834236 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.834929 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.837175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.847165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:51.874259 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1387/2765] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/users => generated 301 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:51.878256 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.892468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ea659-6355-4645-b12b-57b05c60f869 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.897642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.897800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.898281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.898402 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.898568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.898670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.898826 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.898966 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.899658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.899786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.901534 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.901863 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.913692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1388/2766] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:51.924655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.934920 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1379/2767] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:51 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:51.936546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:51.946995 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7b0e-ea24-45fc-98b1-c5b5e24cdfb3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.952101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.952800 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.953220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.953499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] PATH_INFO: `/v3/projects/bb12a587f20a4007a82c292eb652af0b/users/2ec9b7a702f24079bad8fbed3b9c0bec/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.959702 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740c7ce-08c7-44d0-aad2-ac272a3e76a3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:51.967630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:51.970069 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:51.970823 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:51.971296 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:51.973084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.975112 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.985330 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb12a587f20a4007a82c292eb652af0b, user_id=2ec9b7a702f24079bad8fbed3b9c0bec, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:51.988387 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:51.991754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1380/2768] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:51.996482 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.011661 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1389/2769] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:51 2022] PUT /identity/v3/projects/bb12a587f20a4007a82c292eb652af0b/users/2ec9b7a702f24079bad8fbed3b9c0bec/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:52.016581 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.019818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c446-87ab-479c-8abc-f4896e81f187 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.049902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:52.099184 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.099924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.100489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.101200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.101472 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:52.103249 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:52.104812 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaee47e-9fac-412c-b301-61dae106d156 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.113531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.114163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.114379 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1381/2770] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:52.117809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.118114 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.118827 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:52.119248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.120584 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:52.129383 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398b3a-ccae-47b9-816d-a2727a96ff0a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.131509 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1390/2771] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:41:52.135278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3df6c-bab4-4b75-88f1-d63f0ac6d111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.135663 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.136469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3df6c-bab4-4b75-88f1-d63f0ac6d111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.136986 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3df6c-bab4-4b75-88f1-d63f0ac6d111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.137469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3df6c-bab4-4b75-88f1-d63f0ac6d111 admin admin] PATH_INFO: `/v3/projects/ab13c7b859e149dea2577d35a98a549f/users/6d89b871251047b8b4b922a70441477c/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.152068 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e407d5-c14a-40eb-be12-8735b412e301 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.155983 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3df6c-bab4-4b75-88f1-d63f0ac6d111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab13c7b859e149dea2577d35a98a549f, user_id=6d89b871251047b8b4b922a70441477c, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:52.158589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c472c1-aa2f-48ee-8827-607d25a6ae8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.159305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c472c1-aa2f-48ee-8827-607d25a6ae8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.159596 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c472c1-aa2f-48ee-8827-607d25a6ae8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.159705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3df6c-bab4-4b75-88f1-d63f0ac6d111 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:52.159868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c472c1-aa2f-48ee-8827-607d25a6ae8d admin admin] PATH_INFO: `/v3/projects/bb12a587f20a4007a82c292eb652af0b/users/2ec9b7a702f24079bad8fbed3b9c0bec/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.161966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c472c1-aa2f-48ee-8827-607d25a6ae8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb12a587f20a4007a82c292eb652af0b, user_id=2ec9b7a702f24079bad8fbed3b9c0bec, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:52.165017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c472c1-aa2f-48ee-8827-607d25a6ae8d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:52.175432 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1382/2772] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:52 2022] PUT /identity/v3/projects/ab13c7b859e149dea2577d35a98a549f/users/6d89b871251047b8b4b922a70441477c/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:52.191167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1391/2773] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:52 2022] PUT /identity/v3/projects/bb12a587f20a4007a82c292eb652af0b/users/2ec9b7a702f24079bad8fbed3b9c0bec/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:52.196787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.197007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.197201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.230976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] MFA Rules not processed for user `2ec9b7a702f24079bad8fbed3b9c0bec`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:52.232492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:52.243990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.244223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.244945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.275307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] MFA Rules not processed for user `6d89b871251047b8b4b922a70441477c`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:52.276679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:52.317309 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1383/2774] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:52.329528 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.331413 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:52.371863 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1392/2775] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:52.387166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.391797 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:52.417579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd16ac-95bf-4aa4-abf4-26ffb45f00bd None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.421442 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.422087 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.422362 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.422635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.432406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:52.432665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:52.434733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:52.446438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1384/2776] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:41:52.488994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524aafb-e9d5-4dd7-86ad-052702492cd8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.493162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a21e3-d1b0-40cd-817d-378d640607cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf386f112e1b49fdba24871681961e0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.493785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a21e3-d1b0-40cd-817d-378d640607cf service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.494066 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a21e3-d1b0-40cd-817d-378d640607cf service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.494368 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a21e3-d1b0-40cd-817d-378d640607cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.504123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a21e3-d1b0-40cd-817d-378d640607cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:52.505618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a21e3-d1b0-40cd-817d-378d640607cf service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:52.519933 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1393/2777] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:52.600787 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.632497 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:52.675305 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b99a-3451-422e-a115-ce97be014bee service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.680625 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.681386 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.681671 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.681944 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] PATH_INFO: `/v3/users/3aee99dcfc3a4a54abc74404f09ce8a1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.683152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aee99dcfc3a4a54abc74404f09ce8a1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:52.685319 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:52.744575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aee99dcfc3a4a54abc74404f09ce8a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:52.745244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aee99dcfc3a4a54abc74404f09ce8a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:52.745852 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aee99dcfc3a4a54abc74404f09ce8a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:52.762140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aee99dcfc3a4a54abc74404f09ce8a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:52.770350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aee99dcfc3a4a54abc74404f09ce8a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:52.786595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1385/2778] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:52 2022] DELETE /identity/v3/users/3aee99dcfc3a4a54abc74404f09ce8a1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:52.791362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.791849 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.792047 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.811151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:52.813051 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:52.886808 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1394/2779] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:52.898150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:52.899631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:52.982028 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57475567-ee7b-4d4e-a1a1-44e6a58f0d21 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:52.987800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:52.988479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:52.988761 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:52.989034 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:52.999486 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:53.001628 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:53.015654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1386/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:53.312903 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:53.314441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:53.326787 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd54e5-d15c-4075-8708-dad6cefe64a9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:53.332036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:53.332753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:53.333104 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:53.333459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] PATH_INFO: `/v3/projects/da1cd6e7ebc748299fb42f6dab6ea5b0` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:53.334698 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1cd6e7ebc748299fb42f6dab6ea5b0)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:53.337405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:53.388860 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1cd6e7ebc748299fb42f6dab6ea5b0', 'request_id': 'req-f601797e-9588-453c-9b68-815b25c8e096'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.395659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1cd6e7ebc748299fb42f6dab6ea5b0', 'request_id': 'req-f601797e-9588-453c-9b68-815b25c8e096'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.403951 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1cd6e7ebc748299fb42f6dab6ea5b0', 'request_id': 'req-f601797e-9588-453c-9b68-815b25c8e096'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.415881 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] The token cache is being invalidate because project da1cd6e7ebc748299fb42f6dab6ea5b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:53.416173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.416830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.418338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1395/2781] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:53 2022] DELETE /identity/v3/projects/da1cd6e7ebc748299fb42f6dab6ea5b0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:53.422350 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:53.433178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c8b60-71b9-414d-9548-a5ac68c8ab19 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:53.481033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:53.496097 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:53.496660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:53.496937 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:53.497211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] PATH_INFO: `/v3/users/2ec9b7a702f24079bad8fbed3b9c0bec` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:53.498366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec9b7a702f24079bad8fbed3b9c0bec)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:53.500819 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:53.548502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec9b7a702f24079bad8fbed3b9c0bec'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.549157 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec9b7a702f24079bad8fbed3b9c0bec'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.549842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec9b7a702f24079bad8fbed3b9c0bec'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.563795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec9b7a702f24079bad8fbed3b9c0bec'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.570499 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-419f4c6b-cd87-45a6-9f57-ee9eff0fd33c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec9b7a702f24079bad8fbed3b9c0bec'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:53.583541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1387/2782] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:53 2022] DELETE /identity/v3/users/2ec9b7a702f24079bad8fbed3b9c0bec => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:53.859965 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:53.861464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:53.880410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601797e-9588-453c-9b68-815b25c8e096 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:53.913215 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:53.942186 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:53.942838 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:53.943158 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:53.943433 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] PATH_INFO: `/v3/projects/bb12a587f20a4007a82c292eb652af0b` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:53.944651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb12a587f20a4007a82c292eb652af0b)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:53.946627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:53.993991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb12a587f20a4007a82c292eb652af0b', 'request_id': 'req-da9c1dac-40a3-4f58-a836-d7a15246657b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:54.000686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb12a587f20a4007a82c292eb652af0b', 'request_id': 'req-da9c1dac-40a3-4f58-a836-d7a15246657b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:54.008036 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb12a587f20a4007a82c292eb652af0b', 'request_id': 'req-da9c1dac-40a3-4f58-a836-d7a15246657b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:54.018341 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] The token cache is being invalidate because project bb12a587f20a4007a82c292eb652af0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:54.018688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:54.019356 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-da9c1dac-40a3-4f58-a836-d7a15246657b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:54.020908 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1396/2783] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:53 2022] DELETE /identity/v3/projects/bb12a587f20a4007a82c292eb652af0b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:54.026389 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a44f4-21a7-4966-9ff9-374c144461b9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.026993 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a44f4-21a7-4966-9ff9-374c144461b9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.027147 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a44f4-21a7-4966-9ff9-374c144461b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.049059 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-457a44f4-21a7-4966-9ff9-374c144461b9 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:54.050271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-457a44f4-21a7-4966-9ff9-374c144461b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:54.114940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1388/2784] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:54.122973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7019b4-056d-4959-b223-2c01c2bd8756 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.123258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7019b4-056d-4959-b223-2c01c2bd8756 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.123532 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7019b4-056d-4959-b223-2c01c2bd8756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.143957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5b7019b4-056d-4959-b223-2c01c2bd8756 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:54.145144 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5b7019b4-056d-4959-b223-2c01c2bd8756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:54.153685 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1397/2785] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:54.158695 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a44f4-21a7-4966-9ff9-374c144461b9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.168398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a44f4-21a7-4966-9ff9-374c144461b9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.172958 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.173568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.173846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.174116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.174803 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:54.175140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.176709 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.189882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1389/2786] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:54.194128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7019b4-056d-4959-b223-2c01c2bd8756 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.204917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7019b4-056d-4959-b223-2c01c2bd8756 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.209710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.210357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.210638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.210936 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.211616 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.213518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.246220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1398/2787] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:54 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:54.247734 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.258408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9edf66-9c80-4df6-9845-2316e1a7148b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.311701 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:54.323237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.323943 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.324385 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.324667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.325324 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.327330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.349293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1390/2788] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:54 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:54.353397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.366501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d7e9c-3928-406a-90ad-ea975678bed4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.371442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.372058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.372338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.372615 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.373340 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.375443 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.386455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1399/2789] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:54.390547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.401233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da111638-50ca-432f-b884-63fcdb77a362 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.405706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab3488-f44b-4346-8825-0fa51931afff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.406334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab3488-f44b-4346-8825-0fa51931afff admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.406612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab3488-f44b-4346-8825-0fa51931afff admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.406903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab3488-f44b-4346-8825-0fa51931afff admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.407586 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab3488-f44b-4346-8825-0fa51931afff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.409155 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab3488-f44b-4346-8825-0fa51931afff admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.420780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1391/2790] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:54.424754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.435218 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44dd90-efa3-4271-8ad0-e52803c2ff6d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.440868 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.441699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.442010 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.442250 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] PATH_INFO: `/v3/projects/6a98c196638e460d934aed72397897c0/users/0592fe97637146e699e68b41f376df61/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.457153 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a98c196638e460d934aed72397897c0, user_id=0592fe97637146e699e68b41f376df61, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.459792 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.478595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1400/2791] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:54 2022] PUT /identity/v3/projects/6a98c196638e460d934aed72397897c0/users/0592fe97637146e699e68b41f376df61/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:54.490441 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59671aad-f7b1-4cd4-886d-b354071e8abc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.490731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59671aad-f7b1-4cd4-886d-b354071e8abc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.491049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59671aad-f7b1-4cd4-886d-b354071e8abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.518257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-59671aad-f7b1-4cd4-886d-b354071e8abc None None] MFA Rules not processed for user `0592fe97637146e699e68b41f376df61`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:54.519616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-59671aad-f7b1-4cd4-886d-b354071e8abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:54.585060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1392/2792] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:54.589567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.656664 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eeaf35-6288-4065-b71e-1cbfbdf588a0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.662287 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.663135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.663415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.663686 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.664355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.666471 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.672952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:54.697111 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1401/2793] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:54 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:54.701618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59671aad-f7b1-4cd4-886d-b354071e8abc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.712751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59671aad-f7b1-4cd4-886d-b354071e8abc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.770353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.771006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.771293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.771573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.772244 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.774188 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.796098 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1393/2794] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:54 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:54.800638 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.811262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d6e5f-9251-4810-91c6-f2f5373130ae admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.816345 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.816945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.817223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.817492 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.818175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.819772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.829644 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1402/2795] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:54.835811 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.847078 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0a304-9c26-4090-8bdb-f5b5f23f41ed admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.855508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.856175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.856451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.856724 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] PATH_INFO: `/v3/projects/67ac954939364ed7a494d0fd0d17b501/users/a106d6e2e89c465c8d3e2c0156e28dfd/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.872248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ac954939364ed7a494d0fd0d17b501, user_id=a106d6e2e89c465c8d3e2c0156e28dfd, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.875006 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.888422 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1394/2796] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:54 2022] PUT /identity/v3/projects/67ac954939364ed7a494d0fd0d17b501/users/a106d6e2e89c465c8d3e2c0156e28dfd/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:54.892414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.902732 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809c492-a62d-4080-ba45-e71cdf53d3c5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.965043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fedc31-71d6-424e-9ecd-8382b388ab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.965682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fedc31-71d6-424e-9ecd-8382b388ab74 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:54.965972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fedc31-71d6-424e-9ecd-8382b388ab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:54.966247 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fedc31-71d6-424e-9ecd-8382b388ab74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:54.966974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fedc31-71d6-424e-9ecd-8382b388ab74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:54.969073 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fedc31-71d6-424e-9ecd-8382b388ab74 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:54.979659 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1403/2797] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:54.983800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:54.994642 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d6722-5d37-465c-a201-2e124ab12f9a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:54.999259 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:54.999964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:55.000247 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:55.000518 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] PATH_INFO: `/v3/projects/67ac954939364ed7a494d0fd0d17b501/users/a106d6e2e89c465c8d3e2c0156e28dfd/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:55.002502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ac954939364ed7a494d0fd0d17b501, user_id=a106d6e2e89c465c8d3e2c0156e28dfd, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:55.005142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:55.015635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1395/2798] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:54 2022] PUT /identity/v3/projects/67ac954939364ed7a494d0fd0d17b501/users/a106d6e2e89c465c8d3e2c0156e28dfd/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:55.079688 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0579-0c3b-40f2-a7f8-585daae966d9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:55.079982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0579-0c3b-40f2-a7f8-585daae966d9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:55.080270 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0579-0c3b-40f2-a7f8-585daae966d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:55.108684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-300b0579-0c3b-40f2-a7f8-585daae966d9 None None] MFA Rules not processed for user `a106d6e2e89c465c8d3e2c0156e28dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:55.110416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-300b0579-0c3b-40f2-a7f8-585daae966d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:55.184679 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1404/2799] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:55.196475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:55.198168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:55.214035 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:55.258011 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd0030-ed23-4a56-bfa8-dabd5cab65c4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:55.261611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:55.262232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:55.262511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:55.262784 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:55.272178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:55.273597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:55.283506 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1396/2800] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:55.358779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0579-0c3b-40f2-a7f8-585daae966d9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:55.368959 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0579-0c3b-40f2-a7f8-585daae966d9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:55.373080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2578b-319a-4020-a7a0-995ea99a7730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:55.373730 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2578b-319a-4020-a7a0-995ea99a7730 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:55.374013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2578b-319a-4020-a7a0-995ea99a7730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:55.374289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2578b-319a-4020-a7a0-995ea99a7730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:55.439780 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:55.442822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2578b-319a-4020-a7a0-995ea99a7730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:55.444476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2578b-319a-4020-a7a0-995ea99a7730 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:55.458122 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1405/2801] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:55.754618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:55.824060 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf61862-fd98-4d63-8b14-c7cf8c4d03c1 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:55.828705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:55.829351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:55.829633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:55.829909 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] PATH_INFO: `/v3/users/0592fe97637146e699e68b41f376df61` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:55.831108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0592fe97637146e699e68b41f376df61)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:55.833019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:55.888162 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0592fe97637146e699e68b41f376df61'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:55.888989 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0592fe97637146e699e68b41f376df61'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:55.889606 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0592fe97637146e699e68b41f376df61'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:55.908135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0592fe97637146e699e68b41f376df61'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:55.916774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0592fe97637146e699e68b41f376df61'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:55.931525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1397/2802] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:55 2022] DELETE /identity/v3/users/0592fe97637146e699e68b41f376df61 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:55.936369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:55.936898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:55.936898 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:55.959681 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:55.961277 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:56.026974 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1406/2803] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:56.037674 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:56.039150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:56.080653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:56.100717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353a4dc-b6c1-4960-9b25-698cfd052a8f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:56.105137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:56.105842 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:56.106192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:56.106465 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:56.116420 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:56.117979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:56.127557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1398/2804] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:56.403447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:56.404955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:56.418978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05261a21-aedd-48a1-9aef-e5cb3211f162 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:56.425214 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:56.426080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:56.426374 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:56.426655 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] PATH_INFO: `/v3/projects/6a98c196638e460d934aed72397897c0` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:56.428020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a98c196638e460d934aed72397897c0)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:56.430141 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:56.480171 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a98c196638e460d934aed72397897c0', 'request_id': 'req-9b078a6e-96a7-4448-b905-666d2d5696da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.486281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a98c196638e460d934aed72397897c0', 'request_id': 'req-9b078a6e-96a7-4448-b905-666d2d5696da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.494040 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a98c196638e460d934aed72397897c0', 'request_id': 'req-9b078a6e-96a7-4448-b905-666d2d5696da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.504880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] The token cache is being invalidate because project 6a98c196638e460d934aed72397897c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:56.505159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.505770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.507302 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1407/2805] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:56 2022] DELETE /identity/v3/projects/6a98c196638e460d934aed72397897c0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:56.511469 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:56.522815 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9813e8-626e-4b3d-88cf-c254ac78629f service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:56.585630 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:56.586293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:56.586571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:56.586846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] PATH_INFO: `/v3/users/a106d6e2e89c465c8d3e2c0156e28dfd` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:56.588159 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a106d6e2e89c465c8d3e2c0156e28dfd)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:56.590084 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:56.641856 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a106d6e2e89c465c8d3e2c0156e28dfd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.642546 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a106d6e2e89c465c8d3e2c0156e28dfd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.643195 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a106d6e2e89c465c8d3e2c0156e28dfd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.657477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a106d6e2e89c465c8d3e2c0156e28dfd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.664337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca3d1a8-7419-460b-9efd-052edc98476c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a106d6e2e89c465c8d3e2c0156e28dfd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:56.678623 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1399/2806] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:56 2022] DELETE /identity/v3/users/a106d6e2e89c465c8d3e2c0156e28dfd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:57.232486 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.234046 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:57.250011 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078a6e-96a7-4448-b905-666d2d5696da admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:57.310071 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:57.310631 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.310949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.311228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] PATH_INFO: `/v3/projects/67ac954939364ed7a494d0fd0d17b501` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.312408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ac954939364ed7a494d0fd0d17b501)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:57.314313 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:57.336585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:57.361209 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ac954939364ed7a494d0fd0d17b501', 'request_id': 'req-81262c1f-9789-42c1-ab41-246e0121b780'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:57.367338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ac954939364ed7a494d0fd0d17b501', 'request_id': 'req-81262c1f-9789-42c1-ab41-246e0121b780'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:57.375994 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ac954939364ed7a494d0fd0d17b501', 'request_id': 'req-81262c1f-9789-42c1-ab41-246e0121b780'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:57.387450 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] The token cache is being invalidate because project 67ac954939364ed7a494d0fd0d17b501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:57.387735 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:57.388371 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81262c1f-9789-42c1-ab41-246e0121b780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:57.389883 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1408/2807] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:57 2022] DELETE /identity/v3/projects/67ac954939364ed7a494d0fd0d17b501 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:57.394751 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147bb2c-1c51-4aae-9190-946eedb75b6d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.395065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147bb2c-1c51-4aae-9190-946eedb75b6d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.395342 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147bb2c-1c51-4aae-9190-946eedb75b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.415835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7147bb2c-1c51-4aae-9190-946eedb75b6d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:57.417016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7147bb2c-1c51-4aae-9190-946eedb75b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:57.467231 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:57.478403 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1400/2808] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:57.486957 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f744ae-2544-440e-b24e-bacf56536bb7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.487238 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f744ae-2544-440e-b24e-bacf56536bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.487508 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f744ae-2544-440e-b24e-bacf56536bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.507385 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c6f744ae-2544-440e-b24e-bacf56536bb7 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:57.508590 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c6f744ae-2544-440e-b24e-bacf56536bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:57.516401 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1409/2809] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:57.520789 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147bb2c-1c51-4aae-9190-946eedb75b6d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.531821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147bb2c-1c51-4aae-9190-946eedb75b6d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:57.536419 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:57.537033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.537311 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.537614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.538272 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:57.538572 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:57.540177 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:57.553086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1401/2810] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:57.557591 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f744ae-2544-440e-b24e-bacf56536bb7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.568192 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f744ae-2544-440e-b24e-bacf56536bb7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:57.572854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:57.573481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.573763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.574043 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.574716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:57.576672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:57.603691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1410/2811] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:57 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:57.608346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.619782 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493846c5-b01c-4802-859e-5c12d50f9dc2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:57.677858 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:57.678481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.678765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.679072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.679728 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:57.681634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:57.713233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1402/2812] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:57 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:57.714634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.725092 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f39aac-e647-4fd8-a994-8db41842f0e9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:57.729771 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:57.730410 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.730690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.730997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.731684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:57.733274 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:57.750549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1411/2813] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:57.755106 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.765347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6742d56-fa0c-4149-b7ec-b5d3d92ffad3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:57.770050 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7e615-bf2b-4e13-b2a9-550cd786476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:57.770659 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7e615-bf2b-4e13-b2a9-550cd786476a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.770974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7e615-bf2b-4e13-b2a9-550cd786476a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.771251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7e615-bf2b-4e13-b2a9-550cd786476a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.771942 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7e615-bf2b-4e13-b2a9-550cd786476a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:57.773510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7e615-bf2b-4e13-b2a9-550cd786476a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:57.784211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1403/2814] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:57.789096 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.803587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead341-128b-4ec1-ae97-4207a87d7564 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:57.808336 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:57.809249 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.809525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.809802 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] PATH_INFO: `/v3/projects/0a13d7251d894e848426ad8971f6442e/users/e97f8da289be47a58569c428052d5f85/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.825372 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a13d7251d894e848426ad8971f6442e, user_id=e97f8da289be47a58569c428052d5f85, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:57.827997 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:57.845694 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1412/2815] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:57 2022] PUT /identity/v3/projects/0a13d7251d894e848426ad8971f6442e/users/e97f8da289be47a58569c428052d5f85/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:41:57.860526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0568-5162-4c61-b430-3362eae3b24d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:57.860526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0568-5162-4c61-b430-3362eae3b24d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:57.860526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0568-5162-4c61-b430-3362eae3b24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:57.889126 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-bf3c0568-5162-4c61-b430-3362eae3b24d None None] MFA Rules not processed for user `e97f8da289be47a58569c428052d5f85`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:57.890355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bf3c0568-5162-4c61-b430-3362eae3b24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:57.954661 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1404/2816] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:57.965801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:57.967563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:58.026554 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e61cb2-f4ce-475b-8a2b-5da543312221 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:58.032042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb95811-6e23-4d13-a311-e1944b956da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:58.032675 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb95811-6e23-4d13-a311-e1944b956da5 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.033016 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb95811-6e23-4d13-a311-e1944b956da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.033296 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb95811-6e23-4d13-a311-e1944b956da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.042502 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb95811-6e23-4d13-a311-e1944b956da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:58.043999 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb95811-6e23-4d13-a311-e1944b956da5 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:58.053642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1413/2817] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:58.076496 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0568-5162-4c61-b430-3362eae3b24d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:58.127339 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:58.139677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0568-5162-4c61-b430-3362eae3b24d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:58.144286 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:58.144957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.145238 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.145510 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] PATH_INFO: `/v3/users/e97f8da289be47a58569c428052d5f85` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.146682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97f8da289be47a58569c428052d5f85)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:58.148636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:58.202248 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97f8da289be47a58569c428052d5f85'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.203445 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97f8da289be47a58569c428052d5f85'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.204070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97f8da289be47a58569c428052d5f85'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.215844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97f8da289be47a58569c428052d5f85'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.223176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97f8da289be47a58569c428052d5f85'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.235870 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1405/2818] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:41:58 2022] DELETE /identity/v3/users/e97f8da289be47a58569c428052d5f85 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:58.240562 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.241076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.241212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.260033 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:58.261233 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:58.316985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:58.336088 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1414/2819] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:58.344467 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:58.345930 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:58.406461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca725f2-8243-4dbf-bc18-9631610bb710 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:58.410070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574490a3-cf36-4f00-8779-e1fb36f1d295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:58.410684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574490a3-cf36-4f00-8779-e1fb36f1d295 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.410995 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574490a3-cf36-4f00-8779-e1fb36f1d295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.411273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574490a3-cf36-4f00-8779-e1fb36f1d295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.420293 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574490a3-cf36-4f00-8779-e1fb36f1d295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:58.422189 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574490a3-cf36-4f00-8779-e1fb36f1d295 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:58.432115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1406/2820] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:41:58.729814 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:58.731524 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:58.744786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8719c-0b01-43b4-a7af-de420b4a2c19 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:58.749530 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:58.750187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.750475 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.750753 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] PATH_INFO: `/v3/projects/0a13d7251d894e848426ad8971f6442e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.751971 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a13d7251d894e848426ad8971f6442e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:58.753890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:58.800639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a13d7251d894e848426ad8971f6442e', 'request_id': 'req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.807394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a13d7251d894e848426ad8971f6442e', 'request_id': 'req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.815917 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a13d7251d894e848426ad8971f6442e', 'request_id': 'req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.826279 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] The token cache is being invalidate because project 0a13d7251d894e848426ad8971f6442e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:41:58.826569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.827206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d7cd1868-1a0d-492c-b49e-6b1c3c2c893d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:41:58.829462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1415/2821] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:41:58 2022] DELETE /identity/v3/projects/0a13d7251d894e848426ad8971f6442e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:58.834418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255277af-012e-45e3-9f31-deccebe3d64d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.835072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255277af-012e-45e3-9f31-deccebe3d64d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.835072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255277af-012e-45e3-9f31-deccebe3d64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.854568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-255277af-012e-45e3-9f31-deccebe3d64d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:58.855804 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-255277af-012e-45e3-9f31-deccebe3d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:58.917176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1407/2822] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:58.924934 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a2e7b-c691-48d6-9392-49b26f33ceaa None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.925225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a2e7b-c691-48d6-9392-49b26f33ceaa None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.925506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a2e7b-c691-48d6-9392-49b26f33ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.946547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a16a2e7b-c691-48d6-9392-49b26f33ceaa None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:58.947822 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a16a2e7b-c691-48d6-9392-49b26f33ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:58.956459 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1416/2823] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:41:58.960777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255277af-012e-45e3-9f31-deccebe3d64d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:58.970768 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255277af-012e-45e3-9f31-deccebe3d64d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:58.975492 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:58.976116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:58.976397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:58.976669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:58.977366 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:41:58.977672 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:58.979289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:59.002702 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a2e7b-c691-48d6-9392-49b26f33ceaa None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:59.003151 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1408/2824] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:41:59.012090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a2e7b-c691-48d6-9392-49b26f33ceaa None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:59.017230 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:59.017842 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:59.018128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:59.018405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:59.019107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:59.021035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:59.051697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1417/2825] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:41:59 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:41:59.056042 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:59.061762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:41:59.069500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b961ec-4970-4e75-8e78-5c51fb97c7c8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:59.127744 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:59.128273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:59.128553 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:59.128827 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:59.129502 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:59.131481 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:59.152822 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1409/2826] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:41:59 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:41:59.157998 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:59.168923 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4ca6f-c143-4538-a3b2-b26873352dbf admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:59.174106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:59.174919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:59.175289 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:59.175740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:59.176651 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:59.178541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:59.189240 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1418/2827] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:59.193269 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:59.205355 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2249dd-7dc0-42c0-b5de-f38526a1cb7c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:59.209972 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf30fb-34bb-4be1-bc62-a21d2f7994c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:59.210575 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf30fb-34bb-4be1-bc62-a21d2f7994c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:59.210853 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf30fb-34bb-4be1-bc62-a21d2f7994c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:59.211160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf30fb-34bb-4be1-bc62-a21d2f7994c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:59.211844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf30fb-34bb-4be1-bc62-a21d2f7994c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:59.213416 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf30fb-34bb-4be1-bc62-a21d2f7994c8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:59.230086 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1410/2828] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:41:59.234076 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:59.244442 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f3ec1-b577-4f6f-a3f6-0ed60276f37f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:59.249175 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:59.249945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:59.250220 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:59.250375 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] PATH_INFO: `/v3/projects/24680e80e38b4b8d9ae40a59e8c71ffa/users/6642cbedc3d7469382f6f4fe40f9f21f/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:59.266761 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24680e80e38b4b8d9ae40a59e8c71ffa, user_id=6642cbedc3d7469382f6f4fe40f9f21f, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:59.269526 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:59.285600 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1419/2829] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:41:59 2022] PUT /identity/v3/projects/24680e80e38b4b8d9ae40a59e8c71ffa/users/6642cbedc3d7469382f6f4fe40f9f21f/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:41:59.297855 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:59.298289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:59.298780 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:59.331725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] MFA Rules not processed for user `6642cbedc3d7469382f6f4fe40f9f21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:41:59.332961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:59.418752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1411/2830] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:41:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 14:41:59.430239 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:41:59.432013 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:41:59.495193 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559cbf5-00aa-4110-ac06-5a1548d53904 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:41:59.498823 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7909fe-99cf-4dae-a19d-75af8a36f47b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:41:59.499468 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7909fe-99cf-4dae-a19d-75af8a36f47b service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:41:59.500265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7909fe-99cf-4dae-a19d-75af8a36f47b service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:41:59.500541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7909fe-99cf-4dae-a19d-75af8a36f47b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:41:59.508617 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7909fe-99cf-4dae-a19d-75af8a36f47b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:41:59.510055 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7909fe-99cf-4dae-a19d-75af8a36f47b service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:41:59.520020 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1420/2831] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:41:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:42:00.522349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:00.524790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:00.606251 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53209b12-6ff2-4a00-8aea-f1961bb9170e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:00.612149 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:00.612762 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:00.613039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:00.613326 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:00.615116 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:00.625265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:00.628085 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:00.630080 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:00.642114 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1412/2832] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:42:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:42:08.444817 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b104834-692a-4cce-8a1d-ad7846b60b10 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:08.445326 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b104834-692a-4cce-8a1d-ad7846b60b10 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:08.445435 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b104834-692a-4cce-8a1d-ad7846b60b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:08.463955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8b104834-692a-4cce-8a1d-ad7846b60b10 None None] MFA Rules not processed for user `418f616114fa430cbc73d8cbe3263c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:08.465148 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8b104834-692a-4cce-8a1d-ad7846b60b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:08.470558 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:08.524157 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1421/2833] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:09.637844 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:09.705140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b03081-2436-484a-9fc8-6cc7eaafe01f service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:09.709729 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:09.710398 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:09.710731 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:09.711061 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] PATH_INFO: `/v3/users/6642cbedc3d7469382f6f4fe40f9f21f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:09.712430 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6642cbedc3d7469382f6f4fe40f9f21f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:09.714617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:09.759633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6642cbedc3d7469382f6f4fe40f9f21f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:09.760276 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6642cbedc3d7469382f6f4fe40f9f21f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:09.760868 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6642cbedc3d7469382f6f4fe40f9f21f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:09.773039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6642cbedc3d7469382f6f4fe40f9f21f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:09.779846 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6642cbedc3d7469382f6f4fe40f9f21f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:09.793636 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1413/2834] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:42:09 2022] DELETE /identity/v3/users/6642cbedc3d7469382f6f4fe40f9f21f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:42:09.798022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:09.798521 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:09.798635 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:09.817091 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:09.818307 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:09.875531 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1422/2835] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:09.886104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:09.887747 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:09.950703 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2682-301e-456d-87ab-405060eaa46c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:09.954963 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:09.955579 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:09.955882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:09.956156 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:09.965371 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:09.966790 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:09.976738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1414/2836] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:42:10.362621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:10.364358 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:10.382300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11db00-a22d-48bf-a9ec-0841c0c04119 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:10.387392 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:10.388048 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:10.388355 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:10.388602 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] PATH_INFO: `/v3/projects/24680e80e38b4b8d9ae40a59e8c71ffa` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:10.389783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24680e80e38b4b8d9ae40a59e8c71ffa)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:10.392190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:10.439380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24680e80e38b4b8d9ae40a59e8c71ffa', 'request_id': 'req-a45f5cf2-867b-436b-939e-ec5250ee83e2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:10.445948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24680e80e38b4b8d9ae40a59e8c71ffa', 'request_id': 'req-a45f5cf2-867b-436b-939e-ec5250ee83e2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:10.453177 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24680e80e38b4b8d9ae40a59e8c71ffa', 'request_id': 'req-a45f5cf2-867b-436b-939e-ec5250ee83e2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:10.464094 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] The token cache is being invalidate because project 24680e80e38b4b8d9ae40a59e8c71ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:42:10.464708 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:10.465416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a45f5cf2-867b-436b-939e-ec5250ee83e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:10.467021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1423/2837] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:42:10 2022] DELETE /identity/v3/projects/24680e80e38b4b8d9ae40a59e8c71ffa => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:42:19.972748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:20.460930 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:29.970289 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:29.971758 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:29.985031 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded48b8-55e5-4f09-ae12-bc781a4e9355 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:30.040821 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:30.041461 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:30.041738 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:30.042039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] PATH_INFO: `/v3/users/6d89b871251047b8b4b922a70441477c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:30.043227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d89b871251047b8b4b922a70441477c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:30.045119 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:30.091109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d89b871251047b8b4b922a70441477c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.091754 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d89b871251047b8b4b922a70441477c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.092353 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d89b871251047b8b4b922a70441477c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.104667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d89b871251047b8b4b922a70441477c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.111899 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d89b871251047b8b4b922a70441477c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.124300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1415/2838] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:42:29 2022] DELETE /identity/v3/users/6d89b871251047b8b4b922a70441477c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:42:30.128948 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:30.129426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:30.129627 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:30.147886 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:30.149067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:30.207507 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1424/2839] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:30.229650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:30.232039 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:30.306179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3173d-4b1a-4a81-9619-315fc148369d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:30.310135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cab686-2332-46c7-b8b1-90916fb65bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:30.310757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab686-2332-46c7-b8b1-90916fb65bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:30.311081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab686-2332-46c7-b8b1-90916fb65bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:30.311380 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab686-2332-46c7-b8b1-90916fb65bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:30.320532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cab686-2332-46c7-b8b1-90916fb65bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:30.321957 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cab686-2332-46c7-b8b1-90916fb65bc9 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:30.331845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1416/2840] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:42:30.764438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:30.766432 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:30.784740 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d2c6-145e-47af-9c55-f174bc79c541 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:30.791205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:30.791205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:30.791205 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:30.791574 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] PATH_INFO: `/v3/projects/ab13c7b859e149dea2577d35a98a549f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:30.792887 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab13c7b859e149dea2577d35a98a549f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:30.795028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:30.848716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab13c7b859e149dea2577d35a98a549f', 'request_id': 'req-a95752eb-eb12-4322-9d4c-9051b334a0ba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.855889 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab13c7b859e149dea2577d35a98a549f', 'request_id': 'req-a95752eb-eb12-4322-9d4c-9051b334a0ba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.864156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab13c7b859e149dea2577d35a98a549f', 'request_id': 'req-a95752eb-eb12-4322-9d4c-9051b334a0ba'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.878017 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] The token cache is being invalidate because project ab13c7b859e149dea2577d35a98a549f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:42:30.878304 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.879128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a95752eb-eb12-4322-9d4c-9051b334a0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:42:30.880650 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1425/2841] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:42:30 2022] DELETE /identity/v3/projects/ab13c7b859e149dea2577d35a98a549f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:42:30.886500 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70beefe-8578-4151-af2b-99a796fb5155 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:30.887063 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70beefe-8578-4151-af2b-99a796fb5155 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:30.887556 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70beefe-8578-4151-af2b-99a796fb5155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:30.914679 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e70beefe-8578-4151-af2b-99a796fb5155 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:30.916140 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e70beefe-8578-4151-af2b-99a796fb5155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:30.935862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:30.980946 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1417/2842] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:30.988783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9be29-319c-4689-ac3a-ec7d4f03c20e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:30.989067 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9be29-319c-4689-ac3a-ec7d4f03c20e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:30.989343 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9be29-319c-4689-ac3a-ec7d4f03c20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.009809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-80c9be29-319c-4689-ac3a-ec7d4f03c20e None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:31.011074 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-80c9be29-319c-4689-ac3a-ec7d4f03c20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:31.019645 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1426/2843] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:31.023859 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70beefe-8578-4151-af2b-99a796fb5155 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:31.033508 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70beefe-8578-4151-af2b-99a796fb5155 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:31.038020 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:31.038618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.038923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.039201 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.039882 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:42:31.040180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:31.041772 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:31.053005 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1418/2844] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:42:31.056950 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9be29-319c-4689-ac3a-ec7d4f03c20e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:31.066958 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9be29-319c-4689-ac3a-ec7d4f03c20e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:31.071604 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:31.072207 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.072487 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.072759 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.073416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:31.075357 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:31.104235 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1427/2845] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:42:31 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:42:31.108434 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:31.118477 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ceb31-94a3-4a75-9fcd-8204e80b356e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:31.177487 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:31.178132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.178408 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.178677 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.179372 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:31.181370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:31.203568 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1419/2846] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:42:31 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:42:31.207831 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:31.218162 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a9339-7492-426c-a5cf-81ccd346aadc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:31.222765 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:31.223408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.223683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.223954 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.224634 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:31.226607 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:31.237328 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1428/2847] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:42:31.241299 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:31.252653 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29278-fdda-4068-85c4-929cb6b8877d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:31.257257 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022f4fe-8843-4f5c-89a8-044ce13d9020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:31.257862 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022f4fe-8843-4f5c-89a8-044ce13d9020 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.258142 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022f4fe-8843-4f5c-89a8-044ce13d9020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.258440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022f4fe-8843-4f5c-89a8-044ce13d9020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.259145 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022f4fe-8843-4f5c-89a8-044ce13d9020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:31.260706 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022f4fe-8843-4f5c-89a8-044ce13d9020 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:31.272241 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1420/2848] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:42:31.276570 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:31.287806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506298c-b64d-4b73-be31-5cf1e2381bc0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:31.292525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:31.293260 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.293537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.293806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] PATH_INFO: `/v3/projects/779be602b4054e6389c8ade7752e7ac2/users/164c1e7ad0a74e68b4f4cc4bb3eef50e/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.309164 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=779be602b4054e6389c8ade7752e7ac2, user_id=164c1e7ad0a74e68b4f4cc4bb3eef50e, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:31.311885 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:31.325257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1429/2849] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:42:31 2022] PUT /identity/v3/projects/779be602b4054e6389c8ade7752e7ac2/users/164c1e7ad0a74e68b4f4cc4bb3eef50e/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:42:31.330935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396a31f-d28e-4a3a-8a28-25f008141c68 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.331388 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396a31f-d28e-4a3a-8a28-25f008141c68 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.331907 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396a31f-d28e-4a3a-8a28-25f008141c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.353370 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f396a31f-d28e-4a3a-8a28-25f008141c68 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:31.355376 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f396a31f-d28e-4a3a-8a28-25f008141c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:31.417192 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1421/2850] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:31.426851 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:31.428589 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:31.450990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:31.493499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da30d6b-78c9-4891-b80c-ad2177a626aa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:31.497130 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:31.497749 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:31.498029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:31.498301 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:31.507291 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:31.508713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:31.518499 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1430/2851] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:42:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:42:35.346462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cd157-e269-43ca-bc68-930667f217bc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:35.347357 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cd157-e269-43ca-bc68-930667f217bc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:35.347631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cd157-e269-43ca-bc68-930667f217bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:35.376616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-bf8cd157-e269-43ca-bc68-930667f217bc None None] MFA Rules not processed for user `164c1e7ad0a74e68b4f4cc4bb3eef50e`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:35.378109 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bf8cd157-e269-43ca-bc68-930667f217bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:35.398704 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:35.450227 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1422/2852] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:35.455227 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:35.465426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f972f47-866b-4418-946f-54745cfa0d63 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:35.470428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:35.471339 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:35.471619 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:35.471896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:35.472571 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:35.474490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:35.504058 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1431/2853] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:42:35 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:42:35.508070 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cd157-e269-43ca-bc68-930667f217bc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:35.519771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cd157-e269-43ca-bc68-930667f217bc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:35.579178 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:35.579809 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:35.580092 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:35.580367 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:35.581049 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:35.583081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:35.609242 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1423/2854] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:42:35 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:42:35.613490 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:35.625196 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afb99f-2ee8-4040-a8d4-013b9bda4103 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:35.630008 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:35.630640 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:35.630961 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:35.631241 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:35.631940 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:35.633547 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:35.646284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1432/2855] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:42:35.650424 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:35.661612 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f28120-c0a1-43d4-b448-6f15894f0375 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:35.666654 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:35.667404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:35.667686 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:35.667959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] PATH_INFO: `/v3/projects/03d0e7c69b4947e79cab1556e6068f16/users/f54d557adbb246c386d6946801bfab7e/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:35.684340 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d0e7c69b4947e79cab1556e6068f16, user_id=f54d557adbb246c386d6946801bfab7e, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:35.686964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:35.699933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1424/2856] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:42:35 2022] PUT /identity/v3/projects/03d0e7c69b4947e79cab1556e6068f16/users/f54d557adbb246c386d6946801bfab7e/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:42:35.704477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:35.714623 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041d36a-77c3-456a-ba60-8fc190af7824 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:35.772973 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eb6f2-8b8a-4dc5-85d0-295ef3308dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:35.773595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb6f2-8b8a-4dc5-85d0-295ef3308dec admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:35.773879 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb6f2-8b8a-4dc5-85d0-295ef3308dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:35.774151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb6f2-8b8a-4dc5-85d0-295ef3308dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:35.774836 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eb6f2-8b8a-4dc5-85d0-295ef3308dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:35.776515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eb6f2-8b8a-4dc5-85d0-295ef3308dec admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:35.786713 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1433/2857] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:42:35.791104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:35.801641 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25bb7c-89f2-4805-9480-055f5235c9e8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:35.806233 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:35.806913 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:35.807198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:35.807475 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] PATH_INFO: `/v3/projects/03d0e7c69b4947e79cab1556e6068f16/users/f54d557adbb246c386d6946801bfab7e/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:35.809832 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d0e7c69b4947e79cab1556e6068f16, user_id=f54d557adbb246c386d6946801bfab7e, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:35.812489 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:35.822697 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1425/2858] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:42:35 2022] PUT /identity/v3/projects/03d0e7c69b4947e79cab1556e6068f16/users/f54d557adbb246c386d6946801bfab7e/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:42:38.949116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815833b9-e080-499e-8c1d-2b43de5129d3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:38.950143 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815833b9-e080-499e-8c1d-2b43de5129d3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:38.950637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815833b9-e080-499e-8c1d-2b43de5129d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:39.039137 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-815833b9-e080-499e-8c1d-2b43de5129d3 None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:39.043190 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-815833b9-e080-499e-8c1d-2b43de5129d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:39.154039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:39.183778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1434/2859] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Aug 6 14:42:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:39.252638 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:39.266743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:39.448193 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:39.486614 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be6a07-a586-4cec-bb36-c08ee0029c7b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:39.498601 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:39.499778 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:39.500054 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:39.500322 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:39.512866 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:39.516795 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:39.527676 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1426/2860] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:42:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:42:40.868516 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e21fe-98b2-423a-a3bd-0b71a94e56c6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:40.869376 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e21fe-98b2-423a-a3bd-0b71a94e56c6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:40.869737 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e21fe-98b2-423a-a3bd-0b71a94e56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:40.913460 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f80e21fe-98b2-423a-a3bd-0b71a94e56c6 None None] MFA Rules not processed for user `f54d557adbb246c386d6946801bfab7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:40.914697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f80e21fe-98b2-423a-a3bd-0b71a94e56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:41.028203 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1435/2861] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:41.047288 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:42:41.051318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:41.194792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b73174-3ebc-4628-a156-b29f463249a5 service swift] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:42:41.202771 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e19fe-190a-4aaa-b313-299e52bacfee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:42:41.203746 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e19fe-190a-4aaa-b313-299e52bacfee service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:41.204218 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e19fe-190a-4aaa-b313-299e52bacfee service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:41.204692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e19fe-190a-4aaa-b313-299e52bacfee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:41.280009 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e19fe-190a-4aaa-b313-299e52bacfee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:42:41.281610 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e19fe-190a-4aaa-b313-299e52bacfee service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:42:41.302232 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1427/2862] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:42:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:42:42.919273 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2550-8df3-4b66-8d75-926e0375eb28 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:42:42.920159 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2550-8df3-4b66-8d75-926e0375eb28 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:42:42.920642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2550-8df3-4b66-8d75-926e0375eb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:42:42.954447 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d02f2550-8df3-4b66-8d75-926e0375eb28 None None] MFA Rules not processed for user `164c1e7ad0a74e68b4f4cc4bb3eef50e`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:42:42.956140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d02f2550-8df3-4b66-8d75-926e0375eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:42:42.972140 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1436/2863] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:42:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:42:51.292793 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=65 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:42:52.948380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=38 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:01.528749 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:01.529206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:01.529206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:01.549523 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] MFA Rules not processed for user `5f26e831b05740c9a1d510f107e74b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:43:01.550716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:01.557792 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1428/2864] 10.222.0.17 () {58 vars in 967 bytes} [Sat Aug 6 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:43:01.569966 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2550-8df3-4b66-8d75-926e0375eb28 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:01.582528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2550-8df3-4b66-8d75-926e0375eb28 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:01.587537 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add21760f9ed4b82b942c5dd9a6ed096', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:01.588282 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:01.588552 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:01.588828 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:01.597519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:01.599242 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:01.608673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1437/2865] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Aug 6 14:43:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 14:43:03.469150 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:03.470595 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:03.524110 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:03.525587 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:03.549330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e564ce-da98-42ea-aab1-768a1bb8d81d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:03.554265 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:03.555215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:03.555215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:03.555457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:03.557215 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:03.568865 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:03.570359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:03.581082 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1429/2866] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:43:03.594488 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0b5d2-b558-4aae-9ab7-4473ce1a8b59 service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:03.599095 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:03.599955 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:03.600060 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:03.600262 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] PATH_INFO: `/v3/users/418f616114fa430cbc73d8cbe3263c52` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:03.601454 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=418f616114fa430cbc73d8cbe3263c52)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:03.603428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:03.650119 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '418f616114fa430cbc73d8cbe3263c52'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:03.656763 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '418f616114fa430cbc73d8cbe3263c52'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:03.657414 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '418f616114fa430cbc73d8cbe3263c52'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:03.658118 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '418f616114fa430cbc73d8cbe3263c52'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:03.665272 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65d2785d-ff85-43d6-bb4b-4dc33066c023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '418f616114fa430cbc73d8cbe3263c52'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:03.682754 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1438/2867] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:43:03 2022] DELETE /identity/v3/users/418f616114fa430cbc73d8cbe3263c52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:43:03.862448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:03.874682 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c02ff-54c7-4991-ac4b-02678015d8e8 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:03.930605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:03.931281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:03.931555 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:03.931825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] PATH_INFO: `/v3/projects/4bd221dfff634404a071ad30db96b99d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:03.932964 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd221dfff634404a071ad30db96b99d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:03.935223 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:03.979337 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd221dfff634404a071ad30db96b99d', 'request_id': 'req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:03.985418 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd221dfff634404a071ad30db96b99d', 'request_id': 'req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:03.990932 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd221dfff634404a071ad30db96b99d', 'request_id': 'req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:04.003081 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] The token cache is being invalidate because project 4bd221dfff634404a071ad30db96b99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:43:04.003364 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:04.003975 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a9d5fb0c-5964-4f15-8a07-dfbd3a2009ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:43:04.005440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1430/2868] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:43:03 2022] DELETE /identity/v3/projects/4bd221dfff634404a071ad30db96b99d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:43:04.012163 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53f1f-69f7-4c13-8bbe-846a5f4e977d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.012501 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53f1f-69f7-4c13-8bbe-846a5f4e977d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.012774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53f1f-69f7-4c13-8bbe-846a5f4e977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.032281 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-44d53f1f-69f7-4c13-8bbe-846a5f4e977d None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:43:04.033479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-44d53f1f-69f7-4c13-8bbe-846a5f4e977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:04.090123 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1439/2869] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:43:04.097644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d4178-ed3f-4412-9904-c62bd9955463 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.097961 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d4178-ed3f-4412-9904-c62bd9955463 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.098261 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d4178-ed3f-4412-9904-c62bd9955463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.117457 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-763d4178-ed3f-4412-9904-c62bd9955463 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:43:04.118598 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-763d4178-ed3f-4412-9904-c62bd9955463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:04.127292 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1431/2870] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:43:04.131416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53f1f-69f7-4c13-8bbe-846a5f4e977d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:04.141628 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53f1f-69f7-4c13-8bbe-846a5f4e977d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:04.146300 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:04.146935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.147223 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.147500 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.148208 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 14:43:04.148513 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:04.150145 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:04.151624 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:04.163622 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1440/2871] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Aug 6 14:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 14:43:04.167439 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d4178-ed3f-4412-9904-c62bd9955463 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:04.176567 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d4178-ed3f-4412-9904-c62bd9955463 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:04.181440 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:04.182051 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.182334 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.182608 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.183318 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:04.185979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:04.190410 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:04.215498 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1432/2872] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:43:04 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 14:43:04.219642 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:04.229464 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce6b1-927d-4797-8e68-61fd517a7668 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:04.285021 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:04.285637 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.285920 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.286197 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.286859 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:04.288783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:04.310369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1441/2873] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:43:04 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:43:04.315135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:04.325400 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f0b48-9ca8-45e9-9777-ab30d6e5ea78 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:04.329959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:04.330560 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.330837 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.331136 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.331818 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:04.333580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:04.343435 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1433/2874] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 14:43:04.347338 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:04.357134 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051441f-dc2f-4199-8c2e-382a5ca0a62a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:04.362228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d128c0c-6b29-450c-9cdf-32ccbd0813a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:04.362830 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d128c0c-6b29-450c-9cdf-32ccbd0813a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.363151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d128c0c-6b29-450c-9cdf-32ccbd0813a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.363426 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d128c0c-6b29-450c-9cdf-32ccbd0813a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.364106 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d128c0c-6b29-450c-9cdf-32ccbd0813a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:04.365682 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d128c0c-6b29-450c-9cdf-32ccbd0813a3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:04.376801 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1442/2875] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:43:04.380527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:04.390095 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39f17e-3de1-4b9c-b8b2-973a28d29a4c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:04.394603 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:04.395278 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.395557 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.395829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] PATH_INFO: `/v3/projects/f344fdd32c8d482c8df161211d12fcd6/users/0500bcc4b18e4e2cab39fb3d867cce70/roles/471d5a03175b40dd8dc1c327195e2885` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.409757 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f344fdd32c8d482c8df161211d12fcd6, user_id=0500bcc4b18e4e2cab39fb3d867cce70, role_id=471d5a03175b40dd8dc1c327195e2885)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:04.412401 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:04.425365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1434/2876] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:43:04 2022] PUT /identity/v3/projects/f344fdd32c8d482c8df161211d12fcd6/users/0500bcc4b18e4e2cab39fb3d867cce70/roles/471d5a03175b40dd8dc1c327195e2885 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:43:04.430234 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4550-1f62-4fce-a3b2-c58ae359e929 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.430525 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4550-1f62-4fce-a3b2-c58ae359e929 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.430806 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf4550-1f62-4fce-a3b2-c58ae359e929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.450173 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0edf4550-1f62-4fce-a3b2-c58ae359e929 None None] MFA Rules not processed for user `b7c86189492e4a4d8e62d6e2879b6ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:43:04.451415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0edf4550-1f62-4fce-a3b2-c58ae359e929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:04.507978 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1443/2877] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:43:04.517541 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:04.519302 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:04.575994 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c354aef-e583-413e-bc6c-07521e99d5d9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:04.580125 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:04.580739 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:04.581022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:04.581298 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:04.590128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:04.591571 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:04.601525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1435/2878] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:43:08.608949 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ce17-d574-4fc8-b544-ab0b9941c759 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:08.609415 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ce17-d574-4fc8-b544-ab0b9941c759 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:08.609517 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ce17-d574-4fc8-b544-ab0b9941c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:08.642224 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-42c5ce17-d574-4fc8-b544-ab0b9941c759 None None] MFA Rules not processed for user `0500bcc4b18e4e2cab39fb3d867cce70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:43:08.643815 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-42c5ce17-d574-4fc8-b544-ab0b9941c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:08.722515 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1444/2879] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:43:08.727611 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:08.744483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87ed2f-883e-451d-969c-265ff72f03e6 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:08.752526 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:08.753167 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:08.753447 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:08.753721 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:08.754397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:08.756516 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:08.794687 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1436/2880] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Aug 6 14:43:08 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:43:08.798816 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ce17-d574-4fc8-b544-ab0b9941c759 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:08.804397 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:08.811557 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ce17-d574-4fc8-b544-ab0b9941c759 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:08.868298 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:08.868910 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:08.869188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:08.869455 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:08.870116 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:08.872049 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:08.893003 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1445/2881] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Aug 6 14:43:08 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 14:43:08.897003 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:08.907923 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba76de-aa6c-4fbf-bbe9-d9b9e3f49b7e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:08.912536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:08.913160 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:08.913443 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:08.913733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:08.914412 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:08.916075 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:08.926688 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1437/2882] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:43:08.931265 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:08.942286 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc61342-fb3d-44f0-83b0-ea9a36cd12cf admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:08.947124 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:08.947795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:08.948078 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:08.948489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] PATH_INFO: `/v3/projects/86dd84c712214bb69f925064951ab9dc/users/e3903e8b438549e9be2af2516e42aa94/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:08.964952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dd84c712214bb69f925064951ab9dc, user_id=e3903e8b438549e9be2af2516e42aa94, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:08.968395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:08.982009 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1446/2883] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:43:08 2022] PUT /identity/v3/projects/86dd84c712214bb69f925064951ab9dc/users/e3903e8b438549e9be2af2516e42aa94/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:43:08.986158 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:08.996016 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265062b2-e646-4caa-9206-2b1b929e5b26 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:09.050633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:09.051281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:09.051562 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:09.051836 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:09.052540 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:09.054100 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:09.063998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1438/2884] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Aug 6 14:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 14:43:09.068213 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:09.079169 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebd9dd-3030-4812-b228-092b572efb46 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:09.083834 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2927daf-d044-4992-8113-b23873f1fca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:09.084489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927daf-d044-4992-8113-b23873f1fca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:09.084774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927daf-d044-4992-8113-b23873f1fca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:09.085050 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927daf-d044-4992-8113-b23873f1fca8 admin admin] PATH_INFO: `/v3/projects/86dd84c712214bb69f925064951ab9dc/users/e3903e8b438549e9be2af2516e42aa94/roles/93fd86974d4f46edb7f9d9ff528f009d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:09.087041 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2927daf-d044-4992-8113-b23873f1fca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dd84c712214bb69f925064951ab9dc, user_id=e3903e8b438549e9be2af2516e42aa94, role_id=93fd86974d4f46edb7f9d9ff528f009d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:09.089673 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2927daf-d044-4992-8113-b23873f1fca8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:09.101567 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1447/2885] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Aug 6 14:43:09 2022] PUT /identity/v3/projects/86dd84c712214bb69f925064951ab9dc/users/e3903e8b438549e9be2af2516e42aa94/roles/93fd86974d4f46edb7f9d9ff528f009d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:43:11.009524 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:11.011347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:11.032096 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:11.083573 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2d072-f709-4d00-a0d9-fdead5f0e59d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:11.154976 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:11.155924 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:11.156281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:11.156633 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:11.158708 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:11.170527 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:11.171664 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:11.181774 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1439/2886] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:43:15.021406 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c5d3-3a19-4875-bca9-49b969c04ee5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:15.021952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c5d3-3a19-4875-bca9-49b969c04ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:15.021952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c5d3-3a19-4875-bca9-49b969c04ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:15.054980 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d911c5d3-3a19-4875-bca9-49b969c04ee5 None None] MFA Rules not processed for user `e3903e8b438549e9be2af2516e42aa94`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:43:15.056206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d911c5d3-3a19-4875-bca9-49b969c04ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:15.137497 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1448/2887] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:43:15.152004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:15.153885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:15.264199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc2383-c32d-438d-82b0-fcc69c494150 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:15.271462 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:15.272273 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:15.272646 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:15.273017 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:15.342166 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:15.343829 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:15.353719 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1440/2888] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:43:25.133783 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:25.349990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:31.636201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911c5d3-3a19-4875-bca9-49b969c04ee5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:31.645991 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911c5d3-3a19-4875-bca9-49b969c04ee5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:31.649481 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bbad3-aacf-45de-b2c0-49e422a1274c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:31.650080 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bbad3-aacf-45de-b2c0-49e422a1274c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:31.650352 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bbad3-aacf-45de-b2c0-49e422a1274c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:31.650621 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bbad3-aacf-45de-b2c0-49e422a1274c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:31.652394 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-486bbad3-aacf-45de-b2c0-49e422a1274c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:31.718225 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bbad3-aacf-45de-b2c0-49e422a1274c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:31.719692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bbad3-aacf-45de-b2c0-49e422a1274c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:31.729035 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1449/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:43:33.293139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:33.294591 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:33.310692 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca6afd-3568-48de-8b17-d3f77f1dda66 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:33.316224 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:33.316845 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:33.317128 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:33.317402 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:33.319161 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:33.331200 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:33.332777 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:33.344212 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1441/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:43:40.194072 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448789cd-790e-4e90-a5cf-6f36c5581072 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:40.194573 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448789cd-790e-4e90-a5cf-6f36c5581072 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:40.194691 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448789cd-790e-4e90-a5cf-6f36c5581072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:40.213496 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-448789cd-790e-4e90-a5cf-6f36c5581072 None None] MFA Rules not processed for user `0500bcc4b18e4e2cab39fb3d867cce70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:43:40.214672 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-448789cd-790e-4e90-a5cf-6f36c5581072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:40.222113 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1450/2891] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:43:41.667992 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:41.669382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:41.738103 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9603-ee5c-4303-81a2-b19fd00a04ec service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:41.743107 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:41.743658 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:41.743935 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:41.744206 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:41.745921 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:43:41.757065 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:41.758621 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:41.768938 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1442/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:43:50.208258 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=33 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:51.764620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=38 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:43:57.898926 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448789cd-790e-4e90-a5cf-6f36c5581072 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:43:57.909167 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448789cd-790e-4e90-a5cf-6f36c5581072 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:43:57.912805 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:43:57.913408 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:43:57.913683 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:43:57.913953 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:43:57.923527 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:43:57.924938 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:43:57.934322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1451/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:44:07.930109 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:45:11.018124 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:45:11.027616 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4773957-6489-462b-8c3b-cd6b43c3a6f0 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:45:11.031190 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:45:11.031786 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:45:11.032062 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:45:11.032329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:45:11.034053 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:45:11.043239 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:45:11.044645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:45:11.053237 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1443/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:45:21.049536 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:46:00.133261 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:46:00.141919 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe22fe-0a54-428a-aad7-523f4862e96d service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:46:00.145549 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:46:00.146150 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:46:00.146427 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:46:00.146699 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:46:00.148463 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:46:00.162248 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:46:00.163690 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:46:00.175166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1452/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:46:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:46:01.358645 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:46:01.367765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef8267-4baf-4cea-ab36-7042928f2394 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:46:01.371308 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:46:01.371904 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:46:01.372180 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:46:01.372448 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:46:01.374168 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:46:01.388176 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:46:01.389599 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:46:01.400667 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1444/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:46:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:46:10.169630 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:46:11.395765 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:46:13.377428 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:46:13.387648 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c20a-5f8f-4c58-bc9f-0084cebffe3f service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:46:13.392425 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:46:13.395101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:46:13.395697 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:46:13.396188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:46:13.397990 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:46:13.410786 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:46:13.412540 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:46:13.421319 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1453/2897] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:46:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:46:13.533205 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:46:13.542015 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377373b-6d17-4dc5-a60f-c87fcfa6d890 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:46:13.545650 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf760cd-e46a-4abc-b504-552406686bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:46:13.546258 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf760cd-e46a-4abc-b504-552406686bac service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:46:13.546537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf760cd-e46a-4abc-b504-552406686bac service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:46:13.546835 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf760cd-e46a-4abc-b504-552406686bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:46:13.548888 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ddf760cd-e46a-4abc-b504-552406686bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:46:13.562570 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf760cd-e46a-4abc-b504-552406686bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:46:13.564013 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf760cd-e46a-4abc-b504-552406686bac service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:46:13.575211 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1445/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:46:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:46:20.737107 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:46:20.752477 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807c95a-0628-4213-bd84-d865abbc6141 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:46:20.759147 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:46:20.760756 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:46:20.761484 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:46:20.761972 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:46:20.764135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:46:20.778004 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:46:20.779779 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:46:20.792188 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1454/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:46:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:46:21.547605 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c67d0c-dc18-494d-a820-e2bb24a0a412 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:46:21.548135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c67d0c-dc18-494d-a820-e2bb24a0a412 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:46:21.548135 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c67d0c-dc18-494d-a820-e2bb24a0a412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:46:21.568519 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-18c67d0c-dc18-494d-a820-e2bb24a0a412 None None] MFA Rules not processed for user `0500bcc4b18e4e2cab39fb3d867cce70`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:46:21.569725 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-18c67d0c-dc18-494d-a820-e2bb24a0a412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:46:21.576955 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1446/2900] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:46:30.787451 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=12 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:46:31.562417 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=9 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:48:01.942716 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:01.961772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b077-f87d-403a-9be4-896949ed1636 service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:01.965322 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:01.965906 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:01.966182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:01.966452 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:01.968269 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:02.045945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:02.047400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:02.057724 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1455/2901] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:48:07.764511 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c67d0c-dc18-494d-a820-e2bb24a0a412 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:07.778132 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c67d0c-dc18-494d-a820-e2bb24a0a412 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:07.781705 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:07.782312 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:07.782594 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:07.782884 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:07.784597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:07.881004 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:07.882429 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:07.901959 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1447/2902] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:48:10.975707 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:10.977257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:11.009871 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbac97-4f9a-46ae-aa9c-049f196b94bd service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:11.016042 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:11.016781 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:11.017135 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:11.017441 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:11.020023 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:11.033519 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:11.035986 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:11.052416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1456/2903] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:48:13.980998 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:13.982479 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:13.994759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34433c-53e2-498f-8c73-621536c3ce43 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:13.999345 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:14.000030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:14.000300 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:14.000578 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] PATH_INFO: `/v3/users/164c1e7ad0a74e68b4f4cc4bb3eef50e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:14.001748 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=164c1e7ad0a74e68b4f4cc4bb3eef50e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:14.004207 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:14.052894 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164c1e7ad0a74e68b4f4cc4bb3eef50e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.053537 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164c1e7ad0a74e68b4f4cc4bb3eef50e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.054137 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164c1e7ad0a74e68b4f4cc4bb3eef50e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.066596 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164c1e7ad0a74e68b4f4cc4bb3eef50e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.073495 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164c1e7ad0a74e68b4f4cc4bb3eef50e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.085979 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1448/2904] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:48:13 2022] DELETE /identity/v3/users/164c1e7ad0a74e68b4f4cc4bb3eef50e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:48:14.256347 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:14.266890 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6c11a-3221-42c4-b059-3223cbe47cbf service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:14.323166 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:14.323813 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:14.324090 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:14.324361 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] PATH_INFO: `/v3/projects/779be602b4054e6389c8ade7752e7ac2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:14.325510 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=779be602b4054e6389c8ade7752e7ac2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:14.327518 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:14.369559 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:48:14.372569 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '779be602b4054e6389c8ade7752e7ac2', 'request_id': 'req-ad8fdbce-d639-49c6-b6c8-1b852b263356'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.379506 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '779be602b4054e6389c8ade7752e7ac2', 'request_id': 'req-ad8fdbce-d639-49c6-b6c8-1b852b263356'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.386462 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '779be602b4054e6389c8ade7752e7ac2', 'request_id': 'req-ad8fdbce-d639-49c6-b6c8-1b852b263356'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.396902 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] The token cache is being invalidate because project 779be602b4054e6389c8ade7752e7ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:48:14.397252 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.397891 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.399772 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1457/2905] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:48:14 2022] DELETE /identity/v3/projects/779be602b4054e6389c8ade7752e7ac2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:48:14.403525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:14.404927 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:14.419512 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00feb7-824b-42b0-b3bf-3f371915ff2e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:14.445066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:48:14.479426 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:14.480071 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:14.480351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:14.480627 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] PATH_INFO: `/v3/users/f54d557adbb246c386d6946801bfab7e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:14.481808 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54d557adbb246c386d6946801bfab7e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:14.483752 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:14.541665 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54d557adbb246c386d6946801bfab7e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.542330 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54d557adbb246c386d6946801bfab7e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.543033 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54d557adbb246c386d6946801bfab7e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.555666 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54d557adbb246c386d6946801bfab7e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.562620 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54d557adbb246c386d6946801bfab7e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.575597 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1449/2906] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:48:14 2022] DELETE /identity/v3/users/f54d557adbb246c386d6946801bfab7e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:48:14.747305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:14.748785 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:14.761742 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fdbce-d639-49c6-b6c8-1b852b263356 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:14.818059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:14.818719 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:14.819079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:14.819362 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] PATH_INFO: `/v3/projects/03d0e7c69b4947e79cab1556e6068f16` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:14.820505 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d0e7c69b4947e79cab1556e6068f16)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:14.822405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:14.873595 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d0e7c69b4947e79cab1556e6068f16', 'request_id': 'req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.879795 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d0e7c69b4947e79cab1556e6068f16', 'request_id': 'req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.887305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d0e7c69b4947e79cab1556e6068f16', 'request_id': 'req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.898029 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] The token cache is being invalidate because project 03d0e7c69b4947e79cab1556e6068f16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:48:14.898314 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.898967 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d43d9a05-eae5-4a4b-a2a0-110cb31132d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:48:14.900480 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1458/2907] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:48:14 2022] DELETE /identity/v3/projects/03d0e7c69b4947e79cab1556e6068f16 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:48:24.572108 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:48:24.895201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=11,keystone:SELECT=58,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:48:46.279073 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:46.280660 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:46.356730 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0edaa-37e6-4322-abb8-132a43e1016d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:46.403198 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8722d97d3ddc49a082ef84060a30d1b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:46.403800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:46.404077 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:46.404347 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:46.406088 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:46.462911 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:46.464349 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:46.473115 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1450/2908] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:48:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:48:56.469090 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=77 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:48:57.481993 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:57.482541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:57.482541 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:57.501489 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] MFA Rules not processed for user `0500bcc4b18e4e2cab39fb3d867cce70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 14:48:57.502667 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:57.509380 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1459/2909] 10.222.0.17 () {58 vars in 958 bytes} [Sat Aug 6 14:48:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 14:48:57.819397 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:57.820781 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:57.896799 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb098584-6d81-4838-bcf8-1c9238129276 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:57.900382 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:57.900990 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:57.901271 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:57.901544 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:57.903287 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:57.912580 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:57.913974 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:57.922030 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1451/2910] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:48:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:48:58.355556 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:58.356956 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:58.450881 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6ef94-9d6a-4c76-8080-4435bf035a6a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:58.455985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:58.456597 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:58.456878 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:58.457151 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:58.458904 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:58.470028 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:58.471671 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:58.482257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1460/2911] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:48:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:48:58.938618 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:48:58.940072 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:58.954634 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2b7a-685f-4569-86ed-806c78cee9be service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:48:58.959743 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:48:58.960346 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:48:58.960670 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:48:58.961012 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:48:58.962838 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:48:58.974359 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:48:58.975933 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:48:58.985767 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1452/2912] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:48:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:49:08.477845 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=44 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:49:08.981438 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=41 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:49:12.746395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:49:12.747874 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:49:12.858405 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2f576-2cdc-4ac0-b9b8-8765dc87fb49 service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:49:12.864187 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:49:12.864811 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:49:12.865101 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:49:12.865379 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:49:12.867201 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:49:12.879913 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:49:12.881491 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:49:12.893082 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1461/2913] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:49:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:49:15.071558 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:49:15.081019 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e245-e935-463c-a707-afd8909ddab2 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:49:15.084566 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:49:15.085179 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:49:15.085460 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:49:15.085733 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:49:15.087517 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:49:15.152139 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:49:15.153547 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:49:15.163490 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1453/2914] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:49:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:49:22.887985 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=46 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:49:25.159199 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=30 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:50:13.979684 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:50:13.989012 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09e3a6-c9f2-40cb-a25b-4800b712ae4f service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:50:13.992636 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:50:13.993257 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:50:13.993520 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:50:13.993796 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:50:13.995583 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:50:14.005446 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:50:14.006952 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:50:14.014778 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1462/2915] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:50:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:50:24.011264 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:51:06.432052 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:06.433484 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:06.445716 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b563-1c33-4ed4-bc22-23e11e2580f1 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:06.449478 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:06.450240 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:06.450525 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:06.450800 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:06.452635 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:06.466454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:06.467902 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:06.479064 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1454/2916] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:51:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:51:07.666818 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:07.676022 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba4ba-6d24-4423-9ea8-da9846ca209c service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:07.679715 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:07.680350 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:07.680625 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:07.680896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:07.682613 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:07.696568 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:07.697982 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:07.709324 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1463/2917] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:51:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:51:08.676554 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:08.688617 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305847-0fa1-45fe-aee7-9c41e2b5478a service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:08.694529 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '075e9c0ef8fd406087d05a6ce36dc4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:08.695351 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:08.695631 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:08.695903 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:08.697684 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:08.712354 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:08.713918 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:08.725456 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1455/2918] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:51:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 14:51:17.703894 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:51:18.494156 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:18.503027 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454293a-106e-4f16-b3be-38f741bd3766 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:18.506528 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd235744a7ee462eb13e383930de4199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:18.507165 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:18.507438 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:18.507710 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:18.509395 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:18.571618 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:18.573059 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:18.583369 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1464/2919] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:51:18.719898 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=12 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:51:24.536281 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:24.548404 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879025f-55f4-473f-bcd0-a8d7fcd6a541 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:24.554648 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:24.555929 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:24.556220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:24.557007 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:24.559805 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:24.584428 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:24.586152 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:24.596206 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:24.597629 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:24.613479 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441552-97bc-4380-807c-bd8495fdd410 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:24.617940 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1456/2920] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:51:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:51:24.618789 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:24.619476 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:24.619896 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:24.620182 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] PATH_INFO: `/v3/users/0500bcc4b18e4e2cab39fb3d867cce70` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:24.621416 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0500bcc4b18e4e2cab39fb3d867cce70)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:24.623400 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:24.685563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0500bcc4b18e4e2cab39fb3d867cce70'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:24.694305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0500bcc4b18e4e2cab39fb3d867cce70'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:24.695079 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0500bcc4b18e4e2cab39fb3d867cce70'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:24.695770 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0500bcc4b18e4e2cab39fb3d867cce70'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:24.704305 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0500bcc4b18e4e2cab39fb3d867cce70'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:24.723284 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1465/2921] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:51:24 2022] DELETE /identity/v3/users/0500bcc4b18e4e2cab39fb3d867cce70 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 14:51:24.898917 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:24.915980 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c2a32-3878-457a-90d0-d11ff588e8e7 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:24.971669 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:24.972825 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:24.973104 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:24.973377 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] PATH_INFO: `/v3/projects/f344fdd32c8d482c8df161211d12fcd6` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:24.974532 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f344fdd32c8d482c8df161211d12fcd6)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:24.976515 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:25.022027 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f344fdd32c8d482c8df161211d12fcd6', 'request_id': 'req-a94f6d09-6c13-4c0a-939d-64d6bdea2340'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.028022 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f344fdd32c8d482c8df161211d12fcd6', 'request_id': 'req-a94f6d09-6c13-4c0a-939d-64d6bdea2340'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.033454 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f344fdd32c8d482c8df161211d12fcd6', 'request_id': 'req-a94f6d09-6c13-4c0a-939d-64d6bdea2340'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.045715 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] The token cache is being invalidate because project f344fdd32c8d482c8df161211d12fcd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:51:25.046001 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.046644 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.048194 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1457/2922] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:51:24 2022] DELETE /identity/v3/projects/f344fdd32c8d482c8df161211d12fcd6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:51:25.052809 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:25.054221 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:25.068601 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f508479-2d04-4ce2-8f09-8fcee445397d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:25.124019 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:25.124652 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:25.124935 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:25.125212 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] PATH_INFO: `/v3/users/e3903e8b438549e9be2af2516e42aa94` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:25.126363 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3903e8b438549e9be2af2516e42aa94)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:25.128880 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:25.134018 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:51:25.173039 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3903e8b438549e9be2af2516e42aa94'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.180482 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3903e8b438549e9be2af2516e42aa94'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.181458 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3903e8b438549e9be2af2516e42aa94'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.182056 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3903e8b438549e9be2af2516e42aa94'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.189585 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3903e8b438549e9be2af2516e42aa94'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.207228 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1466/2923] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Aug 6 14:51:25 2022] DELETE /identity/v3/users/e3903e8b438549e9be2af2516e42aa94 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:51:25.379582 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:51:25.381175 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:51:25.395483 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f6d09-6c13-4c0a-939d-64d6bdea2340 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:51:25.453885 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c86189492e4a4d8e62d6e2879b6ae2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3509ae96075e47edbbf7f243bb4b1186', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:51:25.454451 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:51:25.454732 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:51:25.455066 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] PATH_INFO: `/v3/projects/86dd84c712214bb69f925064951ab9dc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:51:25.456384 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=86dd84c712214bb69f925064951ab9dc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:51:25.458329 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:51:25.478301 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:51:25.504997 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86dd84c712214bb69f925064951ab9dc', 'request_id': 'req-86313654-dabe-4e04-82c1-6f6196ef3baf'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.511000 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86dd84c712214bb69f925064951ab9dc', 'request_id': 'req-86313654-dabe-4e04-82c1-6f6196ef3baf'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.516365 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86dd84c712214bb69f925064951ab9dc', 'request_id': 'req-86313654-dabe-4e04-82c1-6f6196ef3baf'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.528759 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] The token cache is being invalidate because project 86dd84c712214bb69f925064951ab9dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 14:51:25.529047 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.529662 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86313654-dabe-4e04-82c1-6f6196ef3baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 14:51:25.531220 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1458/2924] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Aug 6 14:51:25 2022] DELETE /identity/v3/projects/86dd84c712214bb69f925064951ab9dc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 14:51:35.203729 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:51:35.525717 dv-cinder-851996-4-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 14:54:14.294639 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 14:54:14.296128 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:54:14.364692 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f446d-d057-4ef5-8614-de62aea4c2d1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 14:54:14.421945 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51e19c-2a84-4ef6-a06b-8b74d6a50388 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd607b8bbb8f047f0bbaf8dad33d47247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '515e01743cac40eca8e54c926c4dff29', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 14:54:14.422563 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51e19c-2a84-4ef6-a06b-8b74d6a50388 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 14:54:14.422844 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51e19c-2a84-4ef6-a06b-8b74d6a50388 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 14:54:14.423139 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51e19c-2a84-4ef6-a06b-8b74d6a50388 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 14:54:14.424854 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8c51e19c-2a84-4ef6-a06b-8b74d6a50388 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 14:54:14.436774 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51e19c-2a84-4ef6-a06b-8b74d6a50388 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 14:54:14.438364 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51e19c-2a84-4ef6-a06b-8b74d6a50388 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 14:54:14.448790 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1467/2925] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Aug 6 14:54:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 14:54:24.444444 dv-cinder-851996-4-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=58 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}